Error Handling

THROUGHPUTS uses standard HTTP status codes and a consistent error shape. Every error includes a machine-readable type and a human-readable message.

Error shape

{
  error: {
    type: "rate_limit_exceeded" | "invalid_request_error" | "authentication_error" |
          "permission_error" | "not_found_error" | "insufficient_quota" |
          "upstream_error" | "server_error";
    message: string;
    code?: string;          // optional provider-specific code
    param?: string;         // optional offending parameter
  };
}

Status codes

Status	Type	Meaning
`400`	`invalid_request_error`	Malformed body, missing required field, or invalid parameter.
`401`	`authentication_error`	Missing, invalid, or revoked API key.
`402`	`insufficient_quota`	Account out of credit or key hit its `spendLimitUsd` cap.
`403`	`permission_error`	Key scope doesn't allow this operation (e.g. `completions-only` key calling `/v1/keys`).
`404`	`not_found_error`	Unknown model slug or endpoint.
`408`	`upstream_error`	Upstream provider timed out; THROUGHPUTS already retried twice.
`429`	`rate_limit_exceeded`	You exceeded your plan's RPM or TPM. See `Retry-After`.
`500`	`server_error`	THROUGHPUTS-side fault. Should be rare.
`502` / `503` / `504`	`upstream_error`	Upstream provider error after THROUGHPUTS retries.

Retryable errors

Retry on these:

Status	When to retry
`408`	Always — upstream timed out.
`429`	After `Retry-After` seconds.
`500`	Always — transient.
`502` / `503` / `504`	Always — upstream down or degraded.

Do not retry on 400, 401, 402, 403, 404 — they won't succeed on retry without code changes.

Backoff strategy

Use exponential backoff with full jitter:

import time, random

def with_retry(fn, max_attempts=5, base_delay=0.5):
    for attempt in range(max_attempts):
        try:
            return fn()
        except RateLimitError as e:
            wait = e.retry_after or (base_delay * (2 ** attempt))
            time.sleep(wait + random.uniform(0, 0.5))
        except (UpstreamError, ServerError) as e:
            time.sleep(base_delay * (2 ** attempt) + random.uniform(0, 0.5))
    raise RuntimeError("Max retries exceeded")

The first-party SDKs do this for you — pass max_retries at client init:

client = Throughputs(api_key="...", max_retries=5)

const client = new Throughputs({ apiKey: "...", maxRetries: 5 });

Streaming errors

If an error occurs mid-stream, THROUGHPUTS sends a final chunk with an error field before closing the connection:

data: {"error":{"type":"upstream_error","message":"Provider timeout"}}

Handle it in your stream loop:

for await (const chunk of stream) {
  if (chunk.error) {
    throw new Error(chunk.error.message);
  }
  // process normal chunk
}

Idempotency

Pass an Idempotency-Key header on POST requests to make them safe to retry. If the same key is seen twice, THROUGHPUTS returns the original response instead of re-running the request:

curl https://api.throughputs.dev/v1/chat/completions \
  -H "Authorization: Bearer $THROUGHPUTS_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d '{...}'

Idempotency keys are stored for 24 hours. Within that window, a retried request with the same key returns the cached response — billed at the original amount, not re-billed.

Debugging

Every error response includes a X-Throughputs-Request-Id header. Include this ID when filing a support ticket — it lets us trace the exact request through our logs.

try:
    response = client.chat.completions.create(...)
except ThroughputsError as e:
    request_id = e.request_id  # thp_req_abc123
    # include this in your support ticket