Rate Limits
NanoGPT doesn’t have one global requests-per-second limit. Limits depend on the endpoint, how the request is billed and the account, and defaults can change. Upstream provider capacity also applies. Handle every429 by waiting for the time in Retry-After if the response has one, then retrying with backoff.
If you need sustained high throughput, email support@nano-gpt.com with:
- which model(s) you need
- your target RPS / peak traffic patterns
- whether requests are streaming
Limits you set on an API key
You can cap each API key by day, week and month, in the key’s settings in the dashboard or through the Management API. Each key has its own counters. If a limit isn’t set, that dimension is unlimited for the key.
These are the Management API field names. USD values are dollars.
null means unlimited, and 0 is a real limit.
Limits use calendar windows in UTC, not rolling windows:
Notes:
- Limits count metered usage, so not every HTTP call counts. Reading the model catalog or your balance doesn’t.
- USD limits use an estimate when the request starts, plus spend still being settled. Actual cost can differ, and concurrent requests can go slightly over a limit.
- Input token units can count some models at a multiple. For example, a 2× model counts each input token twice.
- Requests covered by a subscription skip USD limits because no balance is spent. Request and input token limits still apply.
- The subscription allowance is a soft limit, and in-flight requests can go over it. When it runs out, a subscription-only key stops. A subscription-first key moves to balance only if balance use after limits and paid API models are both enabled for the key.
Account-wide monthly limit
Set this in Settings → Payment & notifications. It covers metered API and web spend across your account, including requests made without an API key, and applies on top of per-key limits. Setting it to0 allows no paid metered spend.
When a limit is reached
The API returns429 with type: "rate_limit_error" and a Retry-After header giving the seconds until the limit resets.
Other codes:
Subscribers can opt in to Overflow to pay-as-you-go in their subscription settings. Requests over the subscription limits (16 at once, 60 a minute) run from your balance at normal prices instead of failing.
Endpoint limits
These are the current defaults for the most commonly used routes.
Text inference has further safeguards that depend on billing. Requests covered by a subscription have concurrency and burst limits, and free models have their own per-minute and per-day limits.