Rate limiting basics

One client can make a thousand requests a second. Nothing in HTTP stops them, and your server will try to answer every one.

Rate limiting controls how much traffic a service accepts. It's the difference between an endpoint that survives a bad afternoon and one that falls over because a single script got enthusiastic.

Three problems it solves

Fair use. One client hogging resources means everyone else waits. A limit keeps a single caller from consuming what the whole user base needs, so legitimate traffic still gets served while somebody's runaway loop is being refused.

Security. A login endpoint with no limit is an invitation to guess passwords, thousands at a time. Limits make brute force slow enough to be useless, and they blunt the simpler denial of service attacks.

Resources. Overload makes a service slow before it makes it unavailable, and slow is where users leave. Capping throughput keeps response times predictable for everyone still inside the limit.

Limits are not a DDoS defence on their own

A distributed denial of service arrives from thousands of separate machines, each looking like an ordinary client. Per-address limits barely touch that.

Answering it takes something upstream: geographic filtering, behavioural analysis, or a provider sitting in front of you. Rate limiting is a layer, not the answer.

JunoThree problems it solves The three overlap, which is why one control covers all of them. Fair use, security and resource protection are the same mechanism seen from different angles.

Capping how often anyone can ask is what makes an app hard to abuse and pleasant for everyone else at the same time.

JunoThree problems it solves The limits worth setting first are usually not the global ones. A login endpoint, a password reset, a signup, a search that hits the database hard: those want their own tight limits, well before a general ceiling on everything.

The useful question per endpoint is what a legitimate user could plausibly do in a minute, then leave headroom above it.

JunoThree problems it solves Rate limiting on login has a subtlety worth knowing: limiting by address punishes an office behind one address, while limiting by username lets an attacker lock out any account they choose by failing its login deliberately.

The shape that holds is limiting both, with different budgets, and never locking an account outright on failures alone. Slowing responses works better than blocking here, because it costs an attacker time without handing them a denial of service against your own users.

Refusing versus slowing

Two related controls that get used interchangeably and behave differently.

	Rate limiting	Throttling
What it does	Refuses requests past a ceiling	Slows requests down as the limit approaches
The client sees	An error, usually 429 Too Many Requests	A response, later than usual
Best for	Hard caps you want enforced	Smoothing bursts without breaking clients

Rate limiting refuses past a ceiling. Throttling slows.

Both have their place, and they combine: slow things down as traffic climbs, refuse outright past a hard ceiling. Throttling covers the second half in detail.

JunoRefusing versus slowing A useful way to hold the difference: refusing says no, and slowing says not so fast.

One breaks the client's request. The other makes them wait. Which is kinder depends entirely on who is calling and why.

JunoRefusing versus slowing Throttling is friendlier to a well-behaved client that briefly asked for too much, because their request still succeeds. It's worse against abuse, since holding requests open costs you connections and memory.

So the usual arrangement is throttle the middle and refuse the top: gentle for clients drifting over, firm for anything far past.

JunoRefusing versus slowing A 429 is only useful if the client can act on it, which means Retry-After, plus the RateLimit headers telling them the limit, what's left and when it resets. Without those, a client's only strategy is to retry immediately, which is exactly what you were trying to prevent.

The failure mode to design against is the retry storm: everybody refused at the same instant, everybody retrying at the same instant. Jitter in client backoff and staggered resets are what break that up, and neither happens by accident.

What a limit counts against

A limit needs something to count per. Four candidates, each with different tradeoffs:

• IP address. Available on every request including anonymous ones, and shared by everyone behind an office network or a mobile carrier.
• Authenticated user id. Precise and fair, and only exists after login.
• API key. Ideal for machine clients, since a key identifies a caller exactly.
• A claim inside a JWT. Useful when identity already travels in the token.

Identifying clients works through picking between them. The short version: anonymous traffic leaves you with the address, and everything else is better when it's available.

Limits also need a window. Per second, per minute, per hour, and the choice shapes the behaviour as much as the number does.

JunoWhat a limit counts against "A hundred requests" means nothing without two more things: a hundred per what, and a hundred from whom.

Every rate limit is those three decisions together, and changing any one of them changes what the limit actually does.

JunoWhat a limit counts against Window length matters more than people expect. A hundred per hour and two per minute average out the same and behave nothing alike: the first allows a hundred requests in one second and then silence, the second never allows a burst at all.

Short windows smooth traffic. Long windows tolerate bursts. Pick for the traffic you actually want to permit.

JunoWhat a limit counts against Whatever you count against becomes something an attacker can rotate. Addresses come cheap from residential proxy pools; free accounts are cheap unless signup is itself protected; API keys are the exception, since issuing one is a deliberate act.

Which is the honest limit of this control: it makes abuse expensive rather than impossible, and the price is set by how hard your cheapest identifier is to acquire.

Worth deciding early where the limiter's own state lives, too. In-process counters are simple and wrong the moment you run two instances, because each keeps its own count and the effective limit doubles.

Try it

An API has one limit: 1,000 requests per hour per IP address. Work out what it fails to prevent.

1. A user on a university network cannot use the app during term time. Why?
2. An attacker wants 10,000 requests an hour. What does that cost them?
3. A script sends all 1,000 requests in the first two seconds of every hour. Is that within the limit?

Compare your answers

1. The limit counts the wrong thing. Everyone on that network shares one outbound address, so thousands of people are sharing a budget of a thousand. The fix is counting per authenticated user where a user exists, and keeping the address limit only for anonymous traffic.

2. Ten addresses. Residential proxy pools sell those by the thousand for very little. Per-address limits make abuse cost something; they do not make it hard. This is the same reason per-IP limiting barely dents a distributed attack.

3. Entirely within the limit. Nothing says how the thousand requests may be spread, so all of them can arrive at once and the endpoint sees a burst it was never protected from. That's what window length controls, and picking one hour permits a spike a one-minute window would never allow.

Three questions, three different weaknesses, and only the third is about the algorithm. The first two are about what you count and how cheaply it can be replaced.

Where this goes next

A limit needs three decisions: how many, over what window, counted against whom. The algorithm is how you track them.

Fixed window counters is the simplest of those algorithms, and the burst it lets through is what every algorithm after it exists to fix.