Load: rate limits and self-protection

Rate limiting

Per consumer, per route, and by resource. Return a matching status code and a header saying when to retry. An undocumented limit produces integrations that break with no explanation.

Separate protecting against abuse from managing capacity. The first is sharp, the second is flexible.

Protect the core

A queue with a maximum length, early load shedding, and resource isolation so a slow route doesn't drain all the connections. If a heavy report route can choke the login route, you have an architecture problem, not a capacity problem.

Test before, not after

Run a load test on an environment resembling production, and find the point where response time explodes. That point is your capacity, and it's worth knowing before a marketing campaign discovers it for you.