Account & Security
Rate Limits
Every account has limits on how many requests it can send per minute and how many it can have in progress at the same time. Accounts with a dedicated agreement can also have a tokens-per-minute limit.
Default limits
| Limit | Default | Scope |
|---|---|---|
| Requests per minute | 600 | Per account, across all models and API keys, on a 60-second sliding window |
| Concurrent requests per model | 50 | Per account |
| Concurrent requests across all models | 100 | Per account |
| Tokens per minute | No limit | Per account, when agreed for your account |
Concurrency limits apply to text generation: Chat Completions, Responses, and Anthropic Messages. The per-minute limit applies to every inference endpoint.
These are the default limits. Accounts with a dedicated agreement can have different limits, including a tokens-per-minute limit. To discuss higher limits for your workload, contact us.
Concurrent requests
A request counts toward your concurrency limit from the moment it is accepted until its response is complete. For a streaming request, that is when the stream ends.
How many requests per minute your concurrency allows depends on how long each request takes:
| Average request duration | Concurrent requests (one model) | Approximate requests per minute |
|---|---|---|
| 2 seconds | 50 | 600 (per-minute limit applies) |
| 10 seconds | 50 | 300 |
| 30 seconds | 50 | 100 |
Long requests, such as reasoning or agent workloads, are bounded by concurrency. Short requests are bounded by the per-minute limit.
Tokens per minute
When your account has a tokens-per-minute limit, each completed request adds its prompt and completion tokens to a 60-second sliding window.
Tokens are counted when a request finishes, so a request that is already running always completes, even if it takes your account over the limit. New requests are rejected until the tokens used in the window fall back below the limit.
The limit applies to text generation: Chat Completions, Responses, and Anthropic Messages.
Rate limit errors
When a limit is reached, the API returns HTTP 429. Check the Retry-After header for how many seconds to wait before retrying.
| Cause | error.code | Retry-After |
|---|---|---|
| Your account's per-minute limit reached | rate_limit_exceeded | Seconds remaining in the current window. X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset are also set. |
| Too many concurrent requests | concurrency_limit_exceeded | 2. X-Concurrency-Limit gives the limit that was reached. |
| Tokens-per-minute limit reached | tpm_limit_exceeded | Seconds until the current window ends. X-TPM-Limit gives your account's limit. |
| Model temporarily at capacity | model_at_capacity | 5. retry-after-ms: 5000 is also set. No X-RateLimit-* headers: this is not your account's limit. On POST /anthropic/v1/messages it is HTTP 529 overloaded_error. |
| Flex request not started in time | flex_capacity_unavailable | 60. See Service Tiers. |
Handling limits
- Wait for the
Retry-Afterinterval before retrying, and use exponential backoff for repeated 429 responses. - The OpenAI and Anthropic SDKs retry
model_at_capacity(and HTTP 529 onPOST /anthropic/v1/messages) automatically after the indicated delay. - Cap the number of requests your client sends in parallel, for example with a semaphore or a worker pool sized below your concurrency limit.
- A streaming request holds its concurrency slot until the stream ends. Close streams you no longer need.