Does Ocelot support sharing routes and rate limit configurations between its instances if we deploy more than one Ocelot instance?

[MMoussaSURE]

Expected Behavior / New Feature

I want to publish two instance of Ocelot at different servers but the under the same domain
and if the user called my domain /{upstreamRoute} one of Ocelot instances should achieve the request but also the two instances should achieve the rate limit of any one of them called, for example:

  "Routes": [
          {
            "DownstreamPathTemplate": "/product",
            "DownstreamScheme": "https",
             "DownstreamHostAndPorts": [
               {
                 "Host": "products.com",
                 "Port": 44322
               }
             ],
             "UpstreamPathTemplate": "/api/product",
             "UpstreamHttpMethod": [ "POST" ],
             "RouteIsCaseSensitive": true,
             "RateLimitOptions": {
                      "ClientWhitelist": [],
                       "EnableRateLimiting": true,
                       "Period": "60s",
                       "PeriodTimespan": 60,
                       "Limit": 1
                     } 
               }
]
"GlobalConfiguration": {
  "BaseUrl": "https://api.products",
  "RateLimitOptions": {
    "DisableRateLimitHeaders": false,
    "QuotaExceededMessage": "Too Many Requests At Short Duration!",
    "HttpStatusCode": 429
  }
}

What should happen is

• if there is more than one request reach my Ocelot domain during current 60s
• one of ocelot instance should achieve the request and also at the same time the two ocelot instances should know that the remaining allowed request is 0

[raman-m]

The short answer is NO.

Each Ocelot instance counts requests independently for rate limiting due to its internal cache storage. However, if you configure your instance to use a distributed cache, then the rate limiting counters will store values in this distributed cache storage.

Therefore, you should use a distributed cache, as Ocelot does not have the capability to allow one instance to share its internal cache with other instances. Nonetheless, this could be an interesting feature to explore further.

[raman-m]

The second design pertains to the storage of Rate Limiting

The Rate Limiting feature is incorporated into DI via the AddRateLimiting method.
The objective is to substitute the following interface object:

Ocelot/src/Ocelot/DependencyInjection/Features.cs

Line 22 in 8c0180a

.AddSingleton<IRateLimitStorage, MemoryCacheRateLimitStorage>();

Rate limiting storage relies heavily on the IRateLimitStorage interface for saving and reading rate limiting counters to and from persistent storage, which in practice is a memory cache: MemoryCacheRateLimitStorage which inherits from the Microsoft IMemoryCache interface.

There is another undocumented class, DistributedCacheRateLimitStorage:

Ocelot/src/Ocelot/RateLimiting/DistributedCacheRateLimitStorage.cs

Line 12 in 8c0180a

public class DistributedCacheRateLimitStorage : IRateLimitStorage

This class incorporates the Microsoft.Extensions.Caching.Distributed.IDistributedCache object as seen in the provided code segment from the Ocelot library:

Ocelot/src/Ocelot/RateLimiting/DistributedCacheRateLimitStorage.cs

Lines 12 to 16 in 8c0180a

	public class DistributedCacheRateLimitStorage : IRateLimitStorage
	{
	private readonly IDistributedCache _memoryCache;
	public DistributedCacheRateLimitStorage(IDistributedCache memoryCache) => _memoryCache = memoryCache;

Our objectives include:

• Integrating a native IDistributedCache object through native storage driver helpers, see MS docs
• Substituting the IRateLimitStorage object with an actual DistributedCacheRateLimitStorage object

The pseudocode could be structured as follows:

    var builder = WebApplication.CreateBuilder(args);
    builder.Configuration.AddJsonFile("ocelot.json");
    builder.Services.AddOcelot()
        .Services.RemoveAll<Ocelot.RateLimiting.IRateLimitStorage>()
        .AddSingleton(typeof(Ocelot.RateLimiting.IRateLimitStorage), typeof(Ocelot.RateLimiting.DistributedCacheRateLimitStorage));
        // Add Microsoft.Extensions.Caching.Distributed.IDistributedCache services

Please be aware that incorporating the Microsoft.Extensions.Caching.Distributed.IDistributedCache object falls outside the scope of Ocelot's responsibilities.
As a developer, you should consult the excellent official Microsoft documentation:

• Distributed caching in ASP.NET Core | Microsoft Learn

This superb guide will assist you in configuring any distributed persistent storage solutions, including Redis, SQL Server, Azure CosmosDB, and others.

Hope it helps!

[MMoussaSURE]

For how many years have you been developing software using the .NET framework? For how many years have you been using Ocelot as a gateway? What's your full name? And what's your LinkedIn?

i'm mohamed ragab
net developer with 6 years of experience
and that is first time to use ocelot
(i write you from my second github account)
the main one is https://github.com/MohamedRagab10793-design

[MMoussaSURE]

@raman-m wrote on Jun 14, 2024

🆗 Let's research the Ocelot code together...

Thank you for your clearness
This may be useful if i want to store all configurations at distributed cache like (Redis) instead of standard cache (memory cache)

[MMoussaSURE]

@raman-m wrote on Jun 14, 2024

The second design pertains to the storage of Rate Limiting

This what I searching for.
Store rate limit values at distributed cache (Redis) to make all ocelot hosted instances have the same rate limit values at the same time.

I apply that after reading your solution throw the following

I just add Microsoft.Extensions.Caching.StackExchangeRedis package version 8.0.6 and update Ocelot to version 23.3.3
Then configure my services like

context.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = configuration["Redis:ConnectionString"]; 
    options.InstanceName = "BaseCacheKey:OcelotRateLimit:";
});

context.Services.AddOcelot(context.Services.GetConfiguration())
                .AddAdministration("/ocelotAdministration", "secretKey")
                .Services.RemoveAll<IRateLimitStorage>()
                         .AddSingleton(typeof(IRateLimitStorage), typeof(DistributedCacheRateLimitStorage));

@raman-m Is there anything in the above configuration that is inconsistent with my needs?

[raman-m]

Good luck! Once all development is complete, it would be beneficial to demonstrate a real, working solution. Perhaps we could prepare a sample project to publish in our repository.
Do you have anything else to add, Mohamed?