Throttling ASP.NET Web API calls

Many API’s out there, such as GitHub’s API, have a concept called “rate limiting” or “throttling” in place. Rate limiting is used to prevent clients from issuing too many requests over a short amount of time to your API. For example, we can limit anonymous API clients to a maximum of 60 requests per hour whereas we can allow more requests to authenticated clients. But how can we implement this?

Intercepting API calls to enforce throttling

Just like ASP.NET MVC, ASP.NET Web API allows us to write action filters. An action filter is an attribute that you can apply to a controller action, an entire controller and even to all controllers in a project. The attribute modifies the way in which the action is executed by intercepting calls to it. Sound like a great approach, right?

Well… yes! Implementing throttling as an action filter would make sense, although in my opinion it has some disadvantages:

  • We have to implement it as an IAuthorizationFilter to make sure it hooks into the pipeline before most other action filters. This feels kind of dirty but it would do the trick as throttling is some sort of “authorization” to make a number of requests to the API.
  • It gets executed quite late in the overall ASP.NET Web API pipeline. While not a big problem, perhaps we want to skip executing certain portions of code whenever throttling occurs.

So while it makes sense to implement throttling as an action filter, I would prefer plugging it earlier in the pipeline. Luckily for us, ASP.NET Web API also provides the concept of message handlers. They accept an HTTP request and return an HTTP response and plug into the pipeline quite early. Here’s a sample throttling message handler:

1 public class ThrottlingHandler 2 : DelegatingHandler 3 { 4 protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) 5 { 6 var identifier = request.GetClientIpAddress(); 7 8 long currentRequests = 1; 9 long maxRequestsPerHour = 60; 10 11 if (HttpContext.Current.Cache[string.Format("throttling_{0}", identifier)] != null) 12 { 13 currentRequests = (long)System.Web.HttpContext.Current.Cache[string.Format("throttling_{0}", identifier)] + 1; 14 HttpContext.Current.Cache[string.Format("throttling_{0}", identifier)] = currentRequests; 15 } 16 else 17 { 18 HttpContext.Current.Cache.Add(string.Format("throttling_{0}", identifier), currentRequests, 19 null, Cache.NoAbsoluteExpiration, TimeSpan.FromHours(1), 20 CacheItemPriority.Low, null); 21 } 22 23 Task<HttpResponseMessage> response = null; 24 if (currentRequests > maxRequestsPerHour) 25 { 26 response = CreateResponse(request, HttpStatusCode.Conflict, "You are being throttled."); 27 } 28 else 29 { 30 response = base.SendAsync(request, cancellationToken); 31 } 32 33 return response; 34 } 35 36 protected Task<HttpResponseMessage> CreateResponse(HttpRequestMessage request, HttpStatusCode statusCode, string message) 37 { 38 var tsc = new TaskCompletionSource<HttpResponseMessage>(); 39 var response = request.CreateResponse(statusCode); 40 response.ReasonPhrase = message; 41 response.Content = new StringContent(message); 42 tsc.SetResult(response); 43 return tsc.Task; 44 } 45 }

We have to register it as well, which we can do when our application starts:

1 config.MessageHandlers.Add(new ThrottlingHandler());

The throttling handler above isn’t ideal. It’s not very extensible nor does it allow scaling out on a web farm. And it’s bound to being hosted in ASP.NET on IIS. It’s bad! Since there’s already a great project called WebApiContrib, I decided to contribute a better throttling handler to it.

Using the WebApiContrib ThrottlingHandler

The easiest way of using the ThrottlingHandler is by registering it using simple parameters like the following, which throttles every user at 60 requests per hour:

1 config.MessageHandlers.Add(new ThrottlingHandler( 2 new InMemoryThrottleStore(), 3 id => 60, 4 TimeSpan.FromHours(1)));

The IThrottleStore interface stores id + current number of requests. There’s only an in-memory store available but you can easily extend it to write this in a distributed cache or a database.

What’s interesting is we can change how our ThrottlingHandler behaves quite easily. Let’s give a specific IP address a better rate limit:

1 config.MessageHandlers.Add(new ThrottlingHandler( 2 new InMemoryThrottleStore(), 3 id => 4 { 5 if (id == "10.0.0.1") 6 { 7 return 5000; 8 } 9 return 60; 10 }, 11 TimeSpan.FromHours(1)));

Wait… Are you telling me this is all IP based? Well yes, by default. But overriding the ThrottlingHandler allows you to do funky things! Here’s a wireframe:

1 public class MyThrottlingHandler : ThrottlingHandler 2 { 3 // ... 4 5 protected override string GetUserIdentifier(HttpRequestMessage request) 6 { 7 // your user id generation logic here 8 } 9 }

By implementing the GetUserIdentifier method, we can for example return an IP address for unauthenticated users and their username for authenticated users. We can then decide on the throttling quota based on username.

Once using it, the ThrottlingHandler will inject two HTTP headers in every response, informing the client about the rate limit:

image

Enjoy! And do checkout WebApiContrib, it contains almost all extensions to ASP.NET Web API you will ever need!

This is an imported post. It was imported from my old blog using an automated tool and may contain formatting errors and/or broken images.

Leave a Comment

avatar

15 responses

  1. Avatar for Donatas Mačiūnas
    Donatas Mačiūnas May 28th, 2013

    You could use Task.FromResult() instead of creating TaskCompletionSource that would save you some boilerplate.

  2. Avatar for Jon Gallant
    Jon Gallant May 29th, 2013

    Nice! Thanks man. I was literally just researching this on Friday.

  3. Avatar for maartenba
    maartenba May 29th, 2013

    Good hint, thanks :-)

  4. Avatar for maartenba
    maartenba May 29th, 2013

    You&#39re welcome!

  5. Avatar for David Rounick
    David Rounick May 30th, 2013

    Yeah, me too. Thanks a ton.

  6. Avatar for Jalpesh Vadgama
    Jalpesh Vadgama June 25th, 2013

    Nice post!! Thanks for sharing!!

  7. Avatar for Marta Rossi
    Marta Rossi September 4th, 2013

    It seems that InMemoryThrottleStore doesn't use Cache but a simple in memory object. So, if a (anonymous) user will call an API that its entry will remain in server memory forever (until the process is recycled, of course)?

  8. Avatar for Maarten Balliauw
    Maarten Balliauw September 4th, 2013

    Yes, that is correct.

  9. Avatar for Marta Rossi
    Marta Rossi September 4th, 2013

    any way to improve it and free server memory?

    I try something like this (code to add to InMemoryThrottleStore, in VB). The big problem is that from this class I cannot really understand if an item is expired, so I've checking generic items "too old"

    Private _cleanUpTime As New System.Threading.Timer(AddressOf CleanUp, Nothing, TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(15))

    Private Sub CleanUp()

    Dim expired = From te In _throttleStore Where te.Value.PeriodStart.AddHours(1) > Now

    For Each item In expired
    Dim dummy As ThrottleEntry
    _throttleStore.TryRemove(item.Key, dummy)
    Next

    End Sub

  10. Avatar for Maarten Balliauw
    Maarten Balliauw September 4th, 2013

    You can easily switch to using Cache, but it would require a manual compile at the moment.

  11. Avatar for Marta Rossi
    Marta Rossi September 4th, 2013

    I've solved passing Period to MemoryStore during ThrottlingHandler constructor, so using a timer in MemoryStore I can remove items older that the period when throttling may count calls ( Dim expired = From te In _throttleStore Where te.Value.PeriodStartUtc.Add(ObservationPeriod) < Now.ToUniversalTime
    ).
    Faster than switch to .net caching and test all code, and so ASP.NET will not remove entries from cache if ask for memory.

  12. Avatar for Lobo Junior
    Lobo Junior September 11th, 2013

    Good approach, nice post. However, i guess that the appropriated status code should be 429, many requests.
    What do you think? Best Regards.

  13. Avatar for Maarten Balliauw
    Maarten Balliauw September 11th, 2013

    Feel free to do a pull request on GitHub

  14. Avatar for Lenny
    Lenny May 17th, 2014

    This solution has a dependency on ASP.NET pipeline that Web API may not have, especially when it runs as self-hosted application. There is https://throttlewebapi.code... project that might provide the functionality.

  15. Avatar for David Verriere
    David Verriere June 14th, 2017

    Thanks for your post, very useful