TypeScript SDK

npm install throughputs
# or
pnpm add throughputs
# or
yarn add throughputs

Initialize

import { Throughputs } from "throughputs";

const client = new Throughputs({
  apiKey: process.env.THROUGHPUTS_API_KEY!, // thp_live_xxx
});

baseURL defaults to https://api.throughputs.dev/v1. Override it for testing or self-hosted deployments:

const client = new Throughputs({
  apiKey: process.env.THROUGHPUTS_API_KEY!,
  baseURL: "https://api.throughputs.dev/v1",
});

Chat completions

const response = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello" }],
});

console.log(response.choices[0].message.content);
console.log(response.throughputsCostUsd); // first-party field

Every response is fully typed. throughputsCostUsd, throughputsModelServed, and throughputsProvider are added on top of the standard OpenAI shape.

Streaming

const stream = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Count to 5" }],
  stream: true,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}

Fallback chains

If the primary model is down or slow, THROUGHPUTS falls back to backups you specify:

const response = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [...],
  modelFallback: ["claude-3-5-sonnet", "llama-3.1-70b"],
});

console.log(response.throughputsModelServed); // which model actually answered

The routing layer tries each in order. The first one to succeed wins.

Edge runtime

The SDK runs in the browser, Node.js, Bun, Deno, Cloudflare Workers, Vercel Edge, and any standard fetch-based runtime. No Node-only APIs are used.

// app/api/chat/route.ts (Next.js route handler)
import { Throughputs } from "throughputs";

export const runtime = "edge";

const client = new Throughputs({ apiKey: process.env.THROUGHPUTS_API_KEY! });

export async function POST(req: Request) {
  const { messages } = await req.json();
  const stream = await client.chat.completions.create({
    model: "gpt-4o",
    messages,
    stream: true,
  });

  return new Response(stream.toReadableStream(), {
    headers: { "Content-Type": "text/event-stream" },
  });
}

Error handling

import {
  ThroughputsError,
  RateLimitError,
  AuthenticationError,
} from "throughputs";

try {
  const response = await client.chat.completions.create({...});
} catch (e) {
  if (e instanceof AuthenticationError) {
    // invalid or revoked key
  } else if (e instanceof RateLimitError) {
    // 429 — back off and retry
    const wait = e.retryAfter;
  } else if (e instanceof ThroughputsError) {
    console.log(e.statusCode, e.message);
  }
}