An in-depth introduction to Rate Limiting
What is throttling?
Throttling is a technique that ensures that the flow of the data or the requests being sent at the target machine/service/sub-system can be consumed at an acceptable rate.
It is a defensive measure and 3 possible reactions could be
slowing down the incoming requests
rejecting the surplus requests
ignoring the surplus requests
Why do we need throttling in the first place?
to prevent system abuse
to allow the amount of traffic we could handle
control the consumption cost
prevent cascading failures leading to a massive outage
Real-world use-cases for throttling
To prevent catastrophic DDoS attack
When your service is under a DDoS attack the rate limiter acts as your first line of defense that could prevent the surplus request from reaching your system. It would only allow the requests to go through at the configured rate.
To gracefully handle a surge of users
It is possible that your product goes viral and now you are seeing a genuine surge in users. Upon getting a genuine surge in users, the stateful components like databases and caches crash which takes down the entire site.
Rate limiter in this case will help in preventing the entire site from going down; although some users would see some error, like 429- Too many requests- your product will continue to seamlessly work for the other set of users.
Multi-tiered limits
Say, you are running a CICD company and offer 3 tiers of pricing- Tier 1 offers 200 minutes of build time, Tier 2 offers 1000 mins while Tier 3 offers unlimited build time. An internal rate limiter can keep track of the build times consumed by a customer and reject the requests once the limit is hit.
Ensure you are not over-consuming
Say, we are consuming a super expensive third-party API and we want to ensure that we are not using it beyond a certain number otherwise the cost will shoot up. An internal rate limiter can keep a check on this to ensure the surplus request does not go through.
Not overwhelming an unprotected system
Hard deleting from a database is an expensive operation. If we are deleting a huge number of rows from the DB it may severely affect the performance of the DB and hence it is best done in a staggered way. An internal rate limiter can help us streamline the writing by spreading them uniformly across time.
Here's the video of my explaining this in-depth 👇 do check it out
It is a common belief that a rate limiter is always external and is designed to prevent our systems from being abused by the external world, but this is not true. In this video, we understand what throttling is, why we need it in the first place and 5 use cases where external and internal rate limiters are super useful.
Outline:
00:00 Introduction
02:56 What is Throttling?
03:37 What rate limiter does when it gets a surge of requests?
06:39 Why do we need a rate limiter?
10:45 Usecase 1: Preventing catastrophic DDoS Attack
12:20 Usecase 2: Gracefully handling a surge in legitimate users
13:46 Usecase 3: Multi-tiered limits
15:42 Usecase 4: Not overusing an expensive vendor
16:48 Usecase 5: Streamlining deletes to protect an unprotected database
You can also
Subscribe to the YT Channel Asli Engineering
Listen to this on the go on Spotify
Thank you so much for reading 🖖 If you found this helpful, do spread the word about it on social media; it would mean the world to me.
You can also follow me on your favourite social media LinkedIn, and Twitter.
Yours truly,
Arpit
arpitbhayani.me
Until next time, stay awesome :)