flaresolverr / flaresolverrsharp Goto Github PK

View Code? Open in Web Editor NEW

159.0 11.0 29.0 69 KB

FlareSolverr .Net / Proxy server to bypass Cloudflare protection

License: MIT License

C# 100.00%

cloudflare puppeteer proxy chrome chromium api csharp netcore

flaresolverrsharp's Introduction

FlareSolverrSharp

FlareSolverr .Net DelegatingHandler / interceptor. FlareSolverr is a proxy server to bypass Cloudflare protection.

Installation

Full-Featured library:

PM> Install-Package FlareSolverr

Dependencies

.NET Standard 1.3

You need a running FlareSolverr service.

Issues

Cloudflare regularly modifies their protection challenge and improves their bot detection capabilities.

If you notice that the anti-bot page has changed, or if library suddenly stops working, please create a GitHub issue so that I can update the code accordingly.

Before submitting an issue, just be sure that you have the latest version of the library.

Usage

A DelegatingHandler that handles the challenge solution automatically.

A type for HTTP handlers that delegate the processing of HTTP response messages to another handler, called the inner handler.

It checks on every request if the clearance is required or not, if required, it solves the challenge in background then returns the response.

Websites not using Cloudflare will be treated normally. You don't need to configure or call anything further, and you can effectively treat all websites as if they're not protected with anything.

var handler = new ClearanceHandler("http://localhost:8191/")
{
    MaxTimeout = 60000,
    ProxyUrl = "http://127.0.0.1:8888",
    ProxyUsername = "cool_username",
    ProxyPassword = "password4"
};

var client = new HttpClient(handler);
var content = await client.GetStringAsync("https://uam.hitmehard.fun/HIT");
Console.WriteLine(content);

Full example here

Options

FlareSolverr Service API

You have to set the FlareSolverr service API in the ClearanceHandler constructor. If you set an empty or null endpoint, FlareSolverrSharp will be able to detect challenges, but it will not be able to solve them.

Example: http://localhost:8191/

MaxTimeout

Max timeout to solve the challenge.

MaxTimeout should be greater than 15000 (15 seconds) because starting the web browser and solving the challenge takes time.

Example: 60000

ProxyUrl

The ProxyUrl which will be sent to FlareSolverr.

Example: http://127.0.0.1:8888

ProxyUsername

The ProxyUsername which will be sent to FlareSolverr. (Only if ProxyUrl has been set)

Use this if the proxy you're using requires authentication.

ProxyPassword

The ProxyPassword which will be sent to FlareSolverr. (Only if ProxyUrl has been set)

Use this if the proxy you're using requires authentication.

flaresolverrsharp's People

Contributors

Stargazers

Watchers

flaresolverrsharp's Issues

ddos-guard for purr-hollands, MuziekFabriek and Film-Paleis no longer being detected

The other ddos-guard for EbookParadijs is still being detected but Puur-Hollands, MuziekFabriek and Film-Paleis no longer works

Error on 20220311170740 for Puur-Hollands.txtJackett.Common.ExceptionWithConfigData: Login failed: No form found on https://www.puur-hollands.org/login.php using form selector form[action="takelogin.php"]
   at Jackett.Common.Indexers.CardigannIndexer.DoLogin() in D:\a\1\s\src\Jackett.Common\Indexers\CardigannIndexer.cs:line 577
   at Jackett.Common.Indexers.CardigannIndexer.PerformQuery(TorznabQuery query) in D:\a\1\s\src\Jackett.Common\Indexers\CardigannIndexer.cs:line 1504




 
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>DDoS protection | Puur-Hollands.org</title>
<style type="text/css">
    .main_t {height: 100%; width: 100%; border: 0px solid black;}
    .main_t td {text-align: center;}
    .main_t td.info {height: 100%; text-align: center; vertical-align: top; padding-top: 5em;}
	
    .sign { color: #ccc; margin-top: 5em;}
    .sign a { color: #ccc; }
</style>
<script>
function get_current_url() {
    document.write(window.location.href);
    return null;
}
</script>
</head>
<body>
    <table class="main_t">
        <tr>
            <td class="info">
                <h1>DDoS protection actief voor ip <a href="/blacklists/2407:7000:9bc1:4100:f17c:ed10:6b08:f37c" target="_new">2407:7000:9bc1:4100:f17c:ed10:6b08:f37c</a></h1>
                <p>Duurt het laden erg lang, klik op de link om door te gaan.</p>
                <a href=''><script>get_current_url();</script></a>


                <p>U word automatisch doorgestuurd na de juiste pagina over 3 seconds.</p>
                <div id="js_info">
                    <p>Zorg dat u Cookies en JavaScript heeft ingeschakeld, anders kunt u niet aanmelden.</p>
                </div>
            </td>
        </tr>
        <tr>
            <td class="sign"><span style="color: #49C73B;">
                   DDoS protection by </span><span style="color: #49C73B;">Puur-Hollands</span><span style="color: #349EBF;">.org</span><br />

        <noscript>Please enable Javascript to see the requested page.</noscript>
            </td>
        </tr>
    </table>
    <script type="text/javascript">
        document.getElementById('js_info').style.display = 'none';
        var date = new Date();

        days = '180';
        date.setTime(date.getTime() + (days * 24*60*60*1000));

        document.cookie = 'ct_anti_ddos_key' + "=" + escape('db56d634f6beab33862a96fb9cf53f84') + "; expires = " + date.toGMTString() + "; path=/";
        setTimeout(function(){
           window.location.reload(1);
        }, 3000);
    </script>
</body>
</html>

no idea why.
[edit] 3rd edit. as time goes by the other sites are also switching to the newly changed ddos-guard and jackett's flaresolverr is failing to handle them.

System.AggregateException: 'One or more errors occurred. (The cookies provided by FlareSolverr are not valid)'

FlareSolverrException: The cookies provided by FlareSolverr are not valid

Check for `DDOS-GUARD` should be case insensitive

anidex.info returns <title>DDoS-Guard</title>.

FlareSolverrSharp/src/FlareSolverrSharp/ChallengeDetector.cs

Line 43 in 97b5d64

responseHtml.Contains("<title>DDOS-GUARD</title>")) // DDOS-GUARD

new title to detect for flaresolverr recaptcha

http://tracker.fenyarnyek.com/ can generates the following recaptcha challenge page

<!DOCTYPE html>
<html>
    <head>
        <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
        <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
        <title>Bot Verification</title>
        <script>
            function onSubmit() {
                document.getElementById('lsrecaptcha-form').submit();
            }

            var onloadCallback = function() {
                var cont = grecaptcha.render('recaptchadiv', {
                    'sitekey': '6LewU34UAAAAAHvXqFOcQlm8z1MP1xpGAZCYEeZY',
                    'callback': onSubmit,

                });
                grecaptcha.execute(cont);
            };
        </script>
        <style>
            body {
                height: 100%;
            }

            .panel {
                padding: 30px;
                max-width: 425px;
                margin: 10% auto;
                box-shadow: 0 0 2px 2px rgba(0, 0, 0, 0.2);
            }

            .title {
                font-size: 1.5em;
                font-weight: 100;
                margin-top: 10px;
                text-align: center;
            }

            .recaptcha-center {
                margin-top: 35px;
                margin-bottom: 20px;
                margin-left: 13%;
                margin-right: 13%;
                display: block;
            }
        </style>
    </head>
    <body>
        <div class="panel">
            <h3 class="title">Verifying that you are not a robot...</h3>
            <form id="lsrecaptcha-form" method="POST" action="/.lsrecap/recaptcha?">
                <div id="recaptchadiv" class="recaptcha-center"></div>
            </form>
        </div>
        <script src="https://www.recaptcha.net/recaptcha/api.js?onload=onloadCallback&render=explicit" async defer></script>
    </body>
</html>

Configure CI/CD to compile the source code on PRs

not working with docker-compose domain

i have a docker-compose file like this:

    app-proxy:
        image: flaresolverr/flaresolverr:v2.2.4
        restart: always

    app:
        image: app:latest
        restart: always
        environment:
            FLARESOLVERR_ENDPOINT: "http://app-proxy:8191"

the above config does not work and keeps giving me invalid cookie error. the config below works just fine.

    app-proxy:
        image: flaresolverr/flaresolverr:v2.2.4
        restart: always
        ports:
            - 127.0.0.1:8191:8191/tcp

    app:
        image: app:latest
        restart: always
        network_mode: "host" # To access proxy.
        environment:
            FLARESOLVERR_ENDPOINT: "http://127.0.0.1:8191"

is it possible to make it work with the 1st config? thanks.

httpclient timeout

i keep getting:

FlareSolverrSharp.Exceptions.FlareSolverrException: Exception: System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

on my raspberry pi but in my logs i can see the flaresolverr is able to solve the request:

REQ-6 Challenge solved
REQ-6 Response in 253.771 s

Wrong configuration - unit tests failed

Hi,

I have installed FlareSolverr service using docker on my computer.
I ran the unit tests in ClearanceHandlerTests to check if everything works fine, but the tests are failed.

Could you please help me to fix this issue?

Solves problem but throws FlareSolverrException

https://www.tibia.com/community/?subtopic=characters&name=Dummy can be solved but FlareSolverrException is thrown.

The code in ClearanceHandler.cs is:

// Change the cookies in the original request with the cookies provided by FlareSolverr
InjectCookies(request, flareSolverrResponse);
response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);

// Detect if there is a challenge in the response
if (ChallengeDetector.IsClearanceRequired(response))
    throw new FlareSolverrException("The cookies provided by FlareSolverr are not valid");

There is no cookie in flareSolverrResponse.Solution.Cookies

IAFD: FlareSolverrException: The cookies provided by FlareSolverr are not valid

Hello,

I'm currently running FlareSolverr and FlaresolverrSharp v2.0 released a few days ago. I am able to successfully run the Sample program (FlaresolverrSharp.Sample), but when I try to use the program with another cloudflare-protected URL I receive
FlareSolverrException: The cookies provided by FlareSolverr are not valid

On Flaresolverr's side, it appears that it cannot detect cloudflare:

2021-10-23T16:17:38-04:00 INFO REQ-2 Incoming request => POST /v1 body: {"maxTimeout":60000,"cmd":"request.get","url":"http://www.iafd.com/results.asp?searchtype=comprehensive&searchstring=Stoya"}
2021-10-23T16:17:44-04:00 INFO REQ-2 Cloudflare not detected
2021-10-23T16:17:46-04:00 INFO REQ-2 Response in 8.416 s

The response message from Flaresolverr does have the correct HTML, so it appears Flaresolverr isn't running into the same problem the .NET webclient is.

On FlaresolverrSharp's side it does detect a cloudflare response and forwards the request to Flaresolverr. It receives the correct response, and even moves what appear to be cloudflare cookies across to the subsequent request (e.g. the __cf_bm cookie), but the .NET web request still returns the typical 403 Forbidden cloudflare response.

If you want to reproduce the issue, try changing the URL in the sample program to:

public static string ProtectedUrl = "http://www.iafd.com/results.asp?searchtype=comprehensive&searchstring=Stoya";

I could use Flaresolverr as a proxy for every request, but it would be nice if the captured cookies worked in a normal request.

Thanks for any assistance you can give

How to use proxy with FlareSolverr?

Hello, I use FlareSolverr on Windows 10 with FlareSolverSharp and this solution works well for my IP address.
How can I use a proxy with FlareSolverSharp?
When I use the example from the readme file, flaresolver uses my IP address instead of the ProxyURL

var handler = new ClearanceHandler("http://localhost:8191/")
{
    MaxTimeout = 60000,
    ProxyUrl = "http://127.0.0.1:8888" //I need to change the proxy IP to something else
};

var client = new HttpClient(handler);
var content = await client.GetStringAsync("https://uam.hitmehard.fun/HIT");
Console.WriteLine(content);

Proxy error

Hi,

First of all thanks for this awesome project 👍

I'm trying to access Cloudfare protected website through FlareSolverrSharp, here the error on FlareSolverr.exe : NS_ERROR_PROXY_CONNECTION_REFUSED

Ask me if any other information is needed, in advance thank you !

ClearanceHandler overwrite Cookies

Hi!

Following the issue Jackett/Jackett#14511 I found that this function

FlareSolverrSharp/src/FlareSolverrSharp/ClearanceHandler.cs

Line 194 in b6e460f

 private void InjectSetCookieHeader(HttpResponseMessage response, FlareSolverrResponse flareSolverrResponse) 

is overwriting the all cookies obtained here

FlareSolverrSharp/src/FlareSolverrSharp/ClearanceHandler.cs

Line 119 in b6e460f

 response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false); 

So when you get the response, it is no longer valid since it is for another different session.

For the specific case of Jackett, the following is happening:

Make a request to http://wolfmax4k.com/, but cloudflare protection pop up
Make a request to flaresolverr to solve the challenge
Make another request to http://wolfmax4k.com/ but with only the cloudflare cookies, without the other cookies obtained in step 2
Now, it injects the rest of the cookies in the step 2, but the result of step 3 is no longer valid, since the token obtained is for another session, not this one.

Possible solutions:

InjectSetCookieHeader should only overwrite cloudflare cookies
All cookies must be sent in the request

FlareSolverrSharp/src/FlareSolverrSharp/ClearanceHandler.cs

Line 119 in b6e460f

response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);

not just cloudflare ones
Both

Edit:
More info about InjectCookies function,
I dont know what "cookieExcess" is for, but I find it strange the value -98... This causes perfectly valid cookies to expire.

Help needed: Correct way to use FlareSolverrSharp with .net 6/7 dependency injection

Hi,

I'm currently trying to use FlareSolverrSharp in one of my applications. Microsoft recommends to use a IHttpClientFactory for dependency injection. This library doesn't seem compatible with that. I always get this error:
System.InvalidOperationException: The 'InnerHandler' property must be null. 'DelegatingHandler' instances provided to 'HttpMessageHandlerBuilder' must not be reused or cached. Handler: 'FlareSolverrSharp.ClearanceHandler'

Is it possible to use it like I'm currently trying to do?

For China mainland users

I've found the project which supports the .net is great, as I set up it in my env in China, google is not reachable in China mainland normally, and the program shows checks as below, is it possible to skip the checking/testing or if it is required, thanks.