throttlingtroll / throttlingtroll Goto Github PK

View Code? Open in Web Editor NEW

89.0 3.0 3.0 552 KB

Rate limiting/throttling middleware for ASP.NET Core and Azure Functions

License: MIT License

C# 99.76% Dockerfile 0.24%

asp-net-core rate-limiter rate-limiting throttling asp-net middleware azure-functions redis

throttlingtroll's Introduction

ThrottlingTroll

Rate limiting/throttling middleware for ASP.NET Core and Azure Functions.

Install from Nuget:

ASP.NET Core	Azure Functions	Azure Functions with ASP.NET Core Integration
`dotnet add package ThrottlingTroll`	`dotnet add package ThrottlingTroll.AzureFunctions`	`dotnet add package ThrottlingTroll.AzureFunctionsAspNet`

Features

Supports ASP.NET Core, Azure Functions (.NET Isolated) and Azure Functions with ASP.NET Core Integration.
Ingress throttling, aka let your service automatically respond with 429 TooManyRequests to some obtrusive clients.
```
   sequenceDiagram
       Client->>+YourService: #127760;HTTP
       alt limit exceeded?
           YourService-->>Client:❌ 429 TooManyRequests
       else
           YourService-->>-Client:✅ 200 OK
       end
```
Implemented as an ASP.NET Core Middleware (for ASP.NET Core) and as an Azure Functions Middleware (for Azure Functions).

Egress throttling, aka limit the number of calls your code is making against some external endpoint.

   sequenceDiagram
       YourService->>+HttpClient: SendAsync()
       alt limit exceeded?
           HttpClient-->>YourService:❌ 429 TooManyRequests
       else
           HttpClient->>+TheirService: #127760;HTTP
           TheirService-->>-HttpClient:✅ 200 OK
           HttpClient-->>-YourService:✅ 200 OK
       end

Implemented as an HttpClient DelegatingHandler, which produces 429 TooManyRequests response (without making the actual call) when a limit is exceeded.

Propagating 429 TooManyRequests from egress to ingress, aka when your service internally makes an HTTP request which results in 429 TooManyRequests, your service can automatically respond with same 429 TooManyRequests to its calling client.
```
   sequenceDiagram
       Client->>+YourService: #127760;HTTP
       YourService->>+TheirService: #127760;HTTP
       TheirService-->>-YourService:❌ 429 TooManyRequests
       YourService-->>-Client:❌ 429 TooManyRequests
```

Custom response fabrics. For ingress it gives full control on what to return when a request is being throttled, and also allows to implement delayed responses (instead of just returning 429 TooManyRequests):

   sequenceDiagram
       Client->>+YourService: #127760;HTTP
       alt limit exceeded?
           YourService-->>YourService: await Task.Delay(RetryAfter)
           YourService-->>Client:✅ 200 OK
       else
           YourService-->>-Client:✅ 200 OK
       end

For egress it also allows ThrottlingTroll to do automatic retries for you:

   sequenceDiagram
       YourService->>+HttpClient: SendAsync()

       loop while 429 TooManyRequests
           HttpClient->>+TheirService: #127760;HTTP
           TheirService-->>-HttpClient:❌ 429 TooManyRequests
           HttpClient-->>HttpClient: await Task.Delay(RetryAfter)
       end

       HttpClient->>+TheirService: #127760;HTTP
       TheirService-->>-HttpClient:✅ 200 OK
       HttpClient-->>-YourService:✅ 200 OK

Storing rate counters in a distributed cache, making your rate limiting policy consistent across all your computing instances. Supported distributed counter stores are:
And you can implement your own.
Three ways of configuring:
- Statically, aka via appsettings.json/host.json. Simplest.
- Programmatically, at startup. In case you want to parametrize something.
- Reactively. You provide a routine, that fetches limits from wherever, and an IntervalToReloadConfigInSeconds for that routine to be called periodically. Allows to reconfigure rules and limits on-the-fly, without restarting your service.
IdentityIdExtractors, that allow you to limit clients individually, based on their IP-addresses, api-keys, tokens, headers, query strings, claims etc. etc.
CostExtractors, that you can use to assign custom costs to different requests. Default cost is 1, but if some of your requests are heavier than the others, you can assign higher costs to them. Another typical usecase for this would be to arrange different pricing tiers for your service: you set the rate limit to something high - and then "charge" clients differently, based on their pricing tier.

Supported rate limiting algorithms

FixedWindow. No more than PermitLimit requests are allowed in IntervalInSeconds. Here is an illustration for the case of no more than 2 requests per each 8 seconds:

The typical drawback of FixedWindow algorithm is that you'd get request rate bursts at the end of each window. So specifically to cope that we have
SlidingWindow. No more than PermitLimit requests are allowed in IntervalInSeconds, but that interval is split into NumOfBuckets. The main benefit of this algorithm over FixedWindow is that if a client constantly exceedes PermitLimit, it will never get any valid response and will always get 429 TooManyRequests. Here is an illustration for the case of no more than 2 requests per each 8 seconds with 2 buckets:

In other words, with SlidingWindow your service gets a smoother request rate.
Semaphore aka Concurrency Limiter. No more than PermitLimit requests are allowed to be executed concurrently. Here is an illustration for the case of no more than 3 concurrent requests:

If you set Semaphore's PermitLimit to 1 and use RedisCounterStore, then ThrottlingTroll will act as a distributed lock. If you add an IdentityIdExtractor (identifying requests by e.g. a query string parameter), then it will turn into named distributed locks.

Documentation

You can find it in our Wiki.

Most concepts and features are the same for all supported platforms. Things that are specific to each platform are highlighted in the relevant READMEs:

ASP.NET Core	Azure Functions	Azure Functions with ASP.NET Core Integration
How to use with ASP.NET Core	How to use with Azure Functions	How to use with Azure Functions ASP.NET Core Integration

Samples

Full minimalistic sample using ASP.NET Core Minimal API:

using ThrottlingTroll;

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.MapGet("/", () => "Hello ThrottlingTroll!");

// Limiting to 1 request per 2 seconds
app.UseThrottlingTroll(options =>
{
    options.Config = new ThrottlingTrollConfig 
    {
        Rules =
        [
            new ThrottlingTrollRule
            {
                LimitMethod = new FixedWindowRateLimitMethod
                {
                    PermitLimit = 1,
                    IntervalInSeconds = 2
                }
            }
        ]
    };
});

app.Run();

Sample projects that demonstrate all the above concepts:

ASP.NET Core	Azure Functions
ThrottlingTrollSampleWeb	ThrottlingTrollSampleFunction
	ThrottlingTrollSampleAspNetFunction
	ThrottlingTrollSampleDotNet6InProcDurableFunction

Contributing

Is very much welcomed.

throttlingtroll's People

Contributors

Stargazers

Watchers

Forkers

dufton bahmad12 manesiotise

throttlingtroll's Issues

Microsoft.AspNetCore.Http is deprecated, need to get rid of it

See https://www.nuget.org/packages/Microsoft.AspNetCore.Http

Make Rule's 'Method' property comma-separated

so that it can take multiple values

[DistributedCacheCounterStore + Distributed Redis Cache] TTL is rounded to the nearest second, therefore setting interval to 1s does not make sense

, as the counter will not actually be stored and will not take effect.

Workaround: always set the interval to more than 1s. But better just use RedisCounterStore.

How to implement limit per identity?

How to implement limit per identity? Imagine we have 100k clients and different rules (or rules set) per client. The current implementation, as I understand it, requires creating a rule per key and using IdentityId to determine which rule is applicable for the client. In this scenario, I would need to add 100k clients, which probably won't work. I was thinking that instead of using IdentityId, we could create a rule id and in the IdentityIdExtractor, we can decide which rule is applicable for a given id. Converting IdentityIdExtractor to Func<IHttpRequestProxy, class> or Func<IHttpRequestProxy, <string,string> instead of Func<IHttpRequestProxy, string> could solve this. Alternatively, is there an easier way to implement this that I am not seeing?

[Optimization] Cache the fact that a particular limit is exceeded

No point in incrementing counters that are already exceeded. Just need to cache those exceeded ones in memory (can use MemoryCache for that). This promises to save a lot of roundtrips to Redis, and this is especially important under stress.

Issue with redis counter store

Hello,

I'm trying to use redis as a counter store but im getting this error: No service for type 'StackExchange.Redis.IConnectionMultiplexer' has been registered. Im also getting this error on the same project ThrottlingTrollSampleWeb. The only things I have updated is changing the redisConnString to be localhost:6379 and uncommented line 82. Below is what in the Program.cs please let me know if I have missed a step:

using Microsoft.Net.Http.Headers;
using StackExchange.Redis;
using System.Text.Json;
using ThrottlingTroll;
using ThrottlingTroll.CounterStores.Redis;

namespace ThrottlingTrollSampleWeb
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var builder = WebApplication.CreateBuilder(args);

            // Add services to the container.

            builder.Services.AddControllers();
            // Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
            builder.Services.AddEndpointsApiExplorer();
            builder.Services.AddSwaggerGen(options =>
            {
                options.IncludeXmlComments(Path.Combine(AppContext.BaseDirectory, "ThrottlingTrollSampleWeb.xml"));
            });

            // If RedisConnectionString is specified, then using RedisCounterStore.
            // Otherwise the default MemoryCacheCounterStore will be used.
            var redisConnString = "localhost:6379";
            if (!string.IsNullOrEmpty(redisConnString))
            {
                builder.Services.AddSingleton<ICounterStore>(
                    new RedisCounterStore(ConnectionMultiplexer.Connect(redisConnString))
                );
            }

            // <ThrottlingTroll Egress Configuration>

            // Configuring a named HttpClient for egress throttling. Rules and limits taken from appsettings.json
            builder.Services.AddHttpClient("my-throttled-httpclient").AddThrottlingTrollMessageHandler();

            // Configuring a named HttpClient that does automatic retries with respect to Retry-After response header
            builder.Services.AddHttpClient("my-retrying-httpclient").AddThrottlingTrollMessageHandler(options =>
            {
                options.ResponseFabric = async (checkResults, requestProxy, responseProxy, cancelToken) =>
                {
                    var egressResponse = (IEgressHttpResponseProxy)responseProxy;

                    egressResponse.ShouldRetry = true;
                };
            });

            // </ThrottlingTroll Egress Configuration>


            var app = builder.Build();

            // Configure the HTTP request pipeline.
            if (app.Environment.IsDevelopment())
            {
                app.UseSwagger();
                app.UseSwaggerUI();
            }

            app.UseHttpsRedirection();

            app.UseAuthorization();

            app.MapControllers();


            // <ThrottlingTroll Ingress Configuration>

            // Normally you'll configure ThrottlingTroll just once, but it's OK to have multiple 
            // middleware instances, with different settings. We're doing it here for demo purposes only.

            // Simplest form. Loads config from appsettings.json and uses MemoryCacheCounterStore by default.
            app.UseThrottlingTroll();

            // Static programmatic configuration
            app.UseThrottlingTroll(options =>
            {
                // Here is how to enable storing rate counters in Redis. You'll also need to add a singleton IConnectionMultiplexer instance beforehand.
                options.CounterStore = new RedisCounterStore(app.Services.GetRequiredService<IConnectionMultiplexer>());

                options.Config = new ThrottlingTrollConfig
                {
                    Rules = new[]
                    {
                        new ThrottlingTrollRule
                        {
                            UriPattern = "/fixed-window-1-request-per-2-seconds-configured-programmatically",
                            LimitMethod = new FixedWindowRateLimitMethod
                            {
                                PermitLimit = 1,
                                IntervalInSeconds = 2
                            }
                        }
                    },

                    // Specifying UniqueName is needed when multiple services store their
                    // rate limit counters in the same cache instance, to prevent those services
                    // from corrupting each other's counters. Otherwise you can skip it.
                    UniqueName = "MyThrottledService1"
                };
            });

            // Dynamic programmatic configuration. Allows to adjust rules and limits without restarting the service.
            app.UseThrottlingTroll(options =>
            {
                options.GetConfigFunc = async () =>
                {
                    // Loading settings from a custom file. You can instead load them from a database
                    // or from anywhere else.

                    string ruleFileName = Path.Combine(AppContext.BaseDirectory, "my-dynamic-throttling-rule.json");

                    string ruleJson = await File.ReadAllTextAsync(ruleFileName);

                    var rule = JsonSerializer.Deserialize<ThrottlingTrollRule>(ruleJson);

                    return new ThrottlingTrollConfig
                    {
                        Rules = new[] { rule }
                    };
                };

                // The above function will be periodically called every 5 seconds
                options.IntervalToReloadConfigInSeconds = 5;
            });

            // Demonstrates how to use custom response fabrics
            app.UseThrottlingTroll(options =>
            {
                options.Config = new ThrottlingTrollConfig
                {
                    Rules = new[]
                    {
                        new ThrottlingTrollRule
                        {
                            UriPattern = "/fixed-window-1-request-per-2-seconds-response-fabric",
                            LimitMethod = new FixedWindowRateLimitMethod
                            {
                                PermitLimit = 1,
                                IntervalInSeconds = 2
                            }
                        }
                    }
                };

                // Custom response fabric, returns 400 BadRequest + some custom content
                options.ResponseFabric = async (checkResults, requestProxy, responseProxy, requestAborted) =>
                {
                    // Getting the rule that was exceeded and with the biggest RetryAfter value
                    var limitExceededResult = checkResults.OrderByDescending(r => r.RetryAfterInSeconds).FirstOrDefault(r => r.RequestsRemaining < 0);
                    if (limitExceededResult == null)
                    {
                        return;
                    }

                    responseProxy.StatusCode = StatusCodes.Status400BadRequest;

                    responseProxy.SetHttpHeader(HeaderNames.RetryAfter, limitExceededResult.RetryAfterHeaderValue);

                    await responseProxy.WriteAsync("Too many requests. Try again later.");
                };
            });

            // Demonstrates how to delay the response instead of returning 429
            app.UseThrottlingTroll(options =>
            {
                options.Config = new ThrottlingTrollConfig
                {
                    Rules = new[]
                    {
                        new ThrottlingTrollRule
                        {
                            UriPattern = "/fixed-window-1-request-per-2-seconds-delayed-response",
                            LimitMethod = new FixedWindowRateLimitMethod
                            {
                                PermitLimit = 1,
                                IntervalInSeconds = 2
                            }
                        }
                    }
                };

                // Custom response fabric, impedes the normal response for 3 seconds
                options.ResponseFabric = async (checkResults, requestProxy, responseProxy, requestAborted) =>
                {
                    await Task.Delay(TimeSpan.FromSeconds(3));

                    var ingressResponse = (IIngressHttpResponseProxy)responseProxy;
                    ingressResponse.ShouldContinueAsNormal = true;
                };
            });

            // Demonstrates how to use identity extractors
            app.UseThrottlingTroll(options =>
            {
                options.Config = new ThrottlingTrollConfig
                {
                    Rules = new[]
                    {
                        new ThrottlingTrollRule
                        {
                            UriPattern = "/fixed-window-3-requests-per-15-seconds-per-each-api-key",
                            LimitMethod = new FixedWindowRateLimitMethod
                            {
                                PermitLimit = 3,
                                IntervalInSeconds = 15
                            },

                            IdentityIdExtractor = request =>
                            {
                                // Identifying clients by their api-key
                                return ((IIncomingHttpRequestProxy)request).Request.Query["api-key"];
                            }
                        }
                    }
                };
            });

            // Demonstrates Semaphore (Concurrency) rate limiter
            // DON'T TEST IT IN BROWSER, because browsers themselves limit the number of concurrent requests to the same URL.
            app.UseThrottlingTroll(options =>
            {
                options.Config = new ThrottlingTrollConfig
                {
                    Rules = new[]
                    {
                        new ThrottlingTrollRule
                        {
                            UriPattern = "/semaphore-2-concurrent-requests",
                            LimitMethod = new SemaphoreRateLimitMethod
                            {
                                PermitLimit = 2
                            }
                        }
                    }
                };
            });

            /// Demonstrates how to make a named distributed critical section with Semaphore (Concurrency) rate limiter and Identity Extractor.
            /// Query string's 'id' parameter is used as identityId.
            // DON'T TEST IT IN BROWSER, because browsers themselves limit the number of concurrent requests to the same URL.
            app.UseThrottlingTroll(options =>
            {
                options.Config = new ThrottlingTrollConfig
                {
                    Rules = new[]
                    {
                        new ThrottlingTrollRule
                        {
                            UriPattern = "/named-critical-section",
                            LimitMethod = new SemaphoreRateLimitMethod
                            {
                                PermitLimit = 1
                            },

                            // This must be set to something > 0 for responses to be automatically delayed
                            MaxDelayInSeconds = 120,

                            IdentityIdExtractor = request =>
                            {
                                // Identifying clients by their id
                                return ((IIncomingHttpRequestProxy)request).Request.Query["id"];
                            }
                        },
                    }
                };
            });

            // Demonstrates how to make a distributed counter with SemaphoreRateLimitMethod
            app.UseThrottlingTroll(options =>
            {
                options.Config = new ThrottlingTrollConfig
                {
                    Rules = new[]
                    {
                        new ThrottlingTrollRule
                        {
                            UriPattern = "/distributed-counter",
                            LimitMethod = new SemaphoreRateLimitMethod
                            {
                                PermitLimit = 1
                            },

                            // This must be set to something > 0 for responses to be automatically delayed
                            MaxDelayInSeconds = 120,

                            IdentityIdExtractor = request =>
                            {
                                // Identifying counters by their id
                                return ((IIncomingHttpRequestProxy)request).Request.Query["id"];
                            }
                        },
                    }
                };
            });

            // Demonstrates how to use cost extractors
            app.UseThrottlingTroll(options =>
            {
                options.Config = new ThrottlingTrollConfig
                {
                    Rules = new[]
                    {
                        new ThrottlingTrollRule
                        {
                            UriPattern = "/fixed-window-balance-of-10-per-20-seconds",

                            LimitMethod = new FixedWindowRateLimitMethod
                            {
                                PermitLimit = 10,
                                IntervalInSeconds = 20
                            },

                            // Specifying a routine to calculate the cost (weight) of each request
                            CostExtractor = request =>
                            {
                                // Cost comes as a 'cost' query string parameter
                                string? cost = ((IIncomingHttpRequestProxy)request).Request.Query["cost"];

                                return long.TryParse(cost, out long val) ? val : 1;
                            }
                        }
                    }
                };
            });

            // </ThrottlingTroll Ingress Configuration>


            app.Run();
        }
    }
}
![error-in-options-counterstore](https://github.com/ThrottlingTroll/ThrottlingTroll/assets/13834499/5538d3c8-3eae-4bd6-b869-d4d62d4513a8)

Implement Azure Table/Cosmos DB counter store

Especially useful for Functions: can just reuse underlying Storage account, no need for any extra services.

Custom response builders

So that the throttled response can be customized.

Feature Request for Azure Functions: Retrieval of Rate Limiting Values

Recently, I have been working on implementing rate limiting in my Azure functions to better manage incoming request traffic. To achieve this, I have been using your library.

However, during my implementation, I have noticed that it is not possible to retrieve important information related to rate limiting directly from the HTTP response generated by the library. Specifically, I would like to be able to include the following headers in my function responses:

RateLimit-Limit: The limit of allowed requests within a specific time period.
RateLimit-Available: The number of available requests within the current rate limit.
RateLimit-Reset-Time: The moment when the current rate limit will reset.

These headers would be extremely useful for clients and consumers of my functions to better understand and manage their requests and adjust their behavior based on the rate limit status.

Thanks

Bug When 2 rules assigned to same identity or Endpoint.

for example, if user have a limit of 20 per minute and 100 per hour and 1000 per day, so we can setup 3 rules for same identity: [{PermitLimit = 20, IntervalInSeconds = 60}, {PermitLimit = 100, IntervalInSeconds = 360}, {PermitLimit = 1000, IntervalInSeconds = 8640}]

the code loops through each rule as expected, but when it register in memory it uses same key for all rules, which make it hit what ever rule limit is lowest.

Default rule when using IdentityId

Hi, can we have a default rule on a URL when using IdentityId?

For example, I tried using this setup:

Rules = new[]
{
    new ThrottlingTrollRule
    {
        UriPattern = "test$",
        IdentityId = "f411b023-dfb0-4756-9319-7bc24d11a682",
        LimitMethod = new FixedWindowRateLimitMethod
        {
            IntervalInSeconds = 1000,
            PermitLimit = 10
        }
    },
    new ThrottlingTrollRule
    {
        UriPattern = "test$",
        IdentityId = "758a3ff4-0655-4183-9406-20fbc8cb1bcc",
        LimitMethod = new FixedWindowRateLimitMethod
        {
            IntervalInSeconds = 1000,
            PermitLimit = 30
        }
    },
    new ThrottlingTrollRule
    {
        UriPattern = "test$",
        
        LimitMethod = new FixedWindowRateLimitMethod
        {
            IntervalInSeconds = 1000,
            PermitLimit = 1
        }
    }
}

I expected that anything that didn't have the identity of f411b023-dfb0-4756-9319-7bc24d11a682 or 758a3ff4-0655-4183-9406-20fbc8cb1bcc would use the third rule but using this setup everything goes through the first rule.

Implement declarative (attribute-based) configuration

So that rules can be applied to particular controllers/methods/functions by placing a ThrottlingTrollAttribute on them.

Support Functions ASP.NET Core Integration

ASP.NET Core Integration for Azure Functions .NET Isolated has just come out of beta.

A few changes needed to support it:

Another .UseThrottlingTroll() overload, without HostBuilderContext parameter.
Use HttpContext and FunctionContext.GetHttpContext() to access it (instead of HttpRequestData/HttpResponseData).
Maybe something else.

Most likely, it should be a separate NuGet package (to avoid unwanted dependencies).

Implement SQL counter store

Ideally, supporting multiple different RDBMSes, e.g. MSSQL, MySQL etc.

Port to Azure Functions (Isolated)

Azure Functions (isolated .NET worker) can now also have middleware, but that middleware model is specific to Functions.
Need to reimplement and push as a separate package.

ArgumentOutOfRangeException "The absolute expiration value must be in the future." when using DistributedCacheCounterStore + Distributed Redis Cache

This results in counters being not incremented and thus limits not being applied.

Workaround: do not use DistributedCacheCounterStore + Distributed Redis Cache, use RedisCounterStore instead.

Allow specifying IdentityIdExtractor at ThrottlingTrollOptions level as well

So that Rules can come from config file, and yet IdentityIdExtractor can also be specified