Why use another proxy when Flaresolverr is technically already one?
While Flaresolverr provides a proxy solution, it currently lacks support for headers and JSON payloads. To bridge this gap, Flaresolverr Mitm Proxy has been introduced. This utility allows users to include headers and send JSON, enhancing the overall capability of Flaresolverr.
A request will be sent to flaresolverr, as usual, but the proxy url config will be set to point towards flaresolverr-mitm-proxy. What will happen is that the request will be proxied from flaresolverr to flaresolverr-mitm-proxy which will then send it to the website and the returned page will be sent back to flaresolverr for the cloudflare shenanigans
If you want to support my work
There are multiple ways to set up and run the Flaresolverr Mitm Proxy:
-
Docker Compose:
- Refer to the provided
docker-compose.ymlas a sample setup. - Note: When using the docker-compose.yml, avoid mapping ports directly. Instead, use the container image name and port 8080 for proxy settings in Flaresolverr. Ensure they are on the same Docker network. Mapping ports can still be used, I just prefer not to.
- Refer to the provided
-
Docker:
- Pull the Docker image and run. The Mitm proxy operates on port 8080, so remember to map this port.
-
Manual Setup:
- Clone this repository.
- Install dependencies.
- Execute using
mitmdump -s mitm.py.
flaresolverr -> flaresolverr-mitm-proxy -> website
see how to use a proxy example
flaresolverr-> flaresolverr-mitm-proxy -> http proxy (optional) -> website
Headers set via special query parameters.
Example:
$$headers[]=Authorization:mytoken
To set the Authorization header value to mytoken. To add multiple headers, chain them:
https://google.com?$$headers[]=Authorization:mytoken&$$headers[]=otherone:othervalue
This also applies for POST requests (They are also in the query parameters)
Sending cookies is a bit different then headers. Flaresolverr already can pass cookies so flaresolverr-mitm doesn't integrates this, please see the flaresolverr docs for cookies
To send a JSON payload in a POST request, utilize the $$post keyword within the postData parameter.
It takes a base64 encoded JSON string (base64 is needed because of how FlareSolverr works)
Example:
For sending this JSON
{
"test": "nice",
"array": ["lol"],
"nested": {
"yeet": "oof"
},
"number": 4
}You will first need to base64 it.
The base64 representation of this JSON payload is
eyJ0ZXN0IjoibmljZSIsImFycmF5IjpbImxvbCJdLCJuZXN0ZWQiOnsieWVldCI6ICJvb2YifSwibnVtYmVyIjo0fQ==
You then need to pass it in the $$post parameter from the postData of FlareSolverr
$$post=eyJ0ZXN0IjoibmljZSIsImFycmF5IjpbImxvbCJdLCJuZXN0ZWQiOnsieWVldCI6ICJvb2YifSwibnVtYmVyIjo0fQ==
If you need to change $$headers[] and $$post for whatever reasons you are able to do so with
SPECIAL_PREFIX_TOKEN, the default is $$ and if changed it will change the prefix
It is also possible to change the split token for headers
HEADER_SPLIT_TOKEN, the default is : for Authorization:mytoken and can be changed
These can be changed in Docker and Docker compose
For those using the Docker Compose setup:
- Start both Flaresolverr and the Mitm proxy:
docker compose up -d- Make a request to Flaresolverr:
curl -L -X POST 'http://localhost:8191/v1' \
-H 'Content-Type: application/json' \
--data-raw '{
"cmd": "request.post",
"url": "https://httpbin.org/post?$$headers[]=Authorization:mytoken",
"maxTimeout": 60000,
"proxy": {
"url": "flaresolverr-mitm-proxy:8080"
},
"postData": "$$post=eyJ0ZXN0IjoibmljZSIsImFycmF5IjpbImxvbCJdLCJuZXN0ZWQiOnsieWVldCI6ICJvb2YifSwibnVtYmVyIjo0fQ=="
}'Note: It is important here that the proxy.url property is valid and points to the flaresolverr-mitm proxy with the correct port, the one shown in this example matches with the example docker-compose file if the services names haven't been altered and the two services are still in the same docker network. Make sure that it is right, it's crucial for this to work because flaresolverr will proxy the request to the flaresolverr-mitm which will do the changes needed
This request will use httpbin.org which will return the everything from the post so you can see if it worked correctly. The response would look something like:
{
"status": "ok",
"message": "Challenge not detected!",
"startTimestamp": 1691721660615,
"endTimestamp": 1691721665638,
"version": "3.3.2",
"solution": {
"url": "https://httpbin.org/post?$$headers[]=Authorization:mytoken",
"status": 200,
"cookies": [],
"userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36",
"headers": {},
"response": `
<html>
<head>
<meta name="color-scheme" content="light dark">
</head>
<body>
<pre style="word-wrap: break-word; white-space: pre-wrap;">
{
"args": {},
"data": "{\\"test\\": \\"nice\\", \\"array\\": [\\"lol\\"], \\"nested\\": {\\"yeet\\": \\"oof\\"}}",
"files": {},
"form": {},
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9",
"Cache-Control": "max-age=0",
"Content-Length": "61",
"Content-Type": "application/json",
"Host": "httpbin.org",
"Origin": "null",
"Sec-Ch-Ua": "\\"Chromium\\";v=\\"115\\", \\"Not/A)Brand\\";v=\\"99\\"",
"Sec-Ch-Ua-Mobile": "?0",
"Sec-Ch-Ua-Platform": "\\"Linux\\"",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "cross-site",
"Authorization": "mytoken",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36",
"X-Amzn-Trace-Id": "Root=1-64d59fbf-3a7a7e743a2c47bb5ab00d3b"
},
"json": {
"test": "nice",
"array": ["lol"],
"nested": {
"yeet": "oof"
},
"number": 4
},
"url": "https://httpbin.org/post"
}
</pre>
</body>
</html>
`
}
}Note: socks proxy are not currently supported, if it is desired with docker, please open an issue and I will gladly add it, if you are not using docker, you can setup proxychains (linux) to make this works, I have no alternatives for windows for now
see mitmproxy/mitmproxy#211
it is possible to make mitm use an upstream proxy. To make it do so, will depend how you start flaresolverr-mitm
To use a http/https proxy using docker, you need to provide the PROXY env variable in this format http://my-upstream-proxy:port
if using docker-compose, it will need to be passed in the environments properties
for docker cli, it will need to be passed as the -e parameter
If your proxy needs auth, you need to use the PROXY_AUTH env variable to pass in username:password ex: PROXY_AUTH=username:password
When running flaresolverr-mitm proxy, instead of mitmdump -s mitm.py the --mode parameter can be added like so mitmdump -s mitm.py --mode upstream:http://my-upstream-proxy:port
If your proxy needs auth, the --upstream-auth parameter, ex: --upstream-auth username:password