Gateway/Get started/AI SDK

AI SDK

Use Inference Bureau from the Vercel AI SDK: every model your Directives allow, with sessions, end users, tags and model suggestions as provider options.

Install

@inferencebureau/ai-sdk-provider works with AI SDK 7. It reads a project key from IB_API_KEY; see Authentication.

npm install @inferencebureau/ai-sdk-provider ai

# .env
IB_API_KEY=ib_live_…

ib(id) is a chat completions model, so generateText, streamText, tools, structured output and multi-step agents work as they do with any provider. Use a model's catalog id from Models, or one of your team's aliases.

import { inferencebureau as ib } from "@inferencebureau/ai-sdk-provider";
import { generateText, streamText } from "ai";

// Any model your Directives allow: a catalog id, or an alias such as org/fast.
const { text } = await generateText({
  model: ib("anthropic/claude-sonnet-5"),
  prompt: "Write a haiku about invoices.",
});

const result = streamText({
  model: ib("openai/gpt-5-mini"),
  messages,
  tools: { lookupOrder },
});
for await (const delta of result.textStream) process.stdout.write(delta);

Claude on the Messages API

ib.messages(id) sends the same calls to POST /v1/messages through @ai-sdk/anthropic, for Anthropic's own features: extended thinking, prompt caching with cacheControl, and the rest of providerOptions.anthropic. Claude models get the settings @ai-sdk/anthropic has for that model. Other models work here too, converted by the gateway.

Set maxOutputTokens on these calls. Without it, @ai-sdk/anthropic asks for the model's whole output limit (64,000 tokens on Claude Sonnet 4.5, 128,000 on Sonnet 5), and each call reserves that much against your tokens per minute and holds its cost against your balance until it finishes.

import { inferencebureau as ib } from "@inferencebureau/ai-sdk-provider";
import { generateText } from "ai";

const result = await generateText({
  model: ib.messages("anthropic/claude-sonnet-5"),
  messages,
  maxOutputTokens: 8000, // otherwise the model's whole limit is reserved
  providerOptions: {
    // Anthropic's own options, as @ai-sdk/anthropic takes them.
    anthropic: { thinking: { type: "enabled", budgetTokens: 4000 } },
  },
});

Embeddings

ib.embedding(id) works with embed and embedMany on POST /v1/embeddings.

import { inferencebureau as ib } from "@inferencebureau/ai-sdk-provider";
import { embedMany } from "ai";

const { embeddings } = await embedMany({
  model: ib.embedding("openai/text-embedding-3-small"),
  values: chunks,
  providerOptions: { inferencebureau: { tags: ["search-index"] } },
});

Sessions, end users and tags

Inference Bureau's options go in providerOptions.inferencebureau on any call. They become the headers and the ib field the API reads, and are never sent to a provider.

const result = streamText({
  model: ib("anthropic/claude-sonnet-5"),
  messages,
  providerOptions: {
    inferencebureau: {
      session: conversationId,      // ib-session: groups the conversation in Logs
      user: currentUser.id,         // ib-user: the end user, for per-user limits and budgets
      tags: ["support", "beta"],    // ib-tags: cost breakdowns
    },
  },
});

Option	What it does
`session`	Groups a conversation's turns in Logs, and is what mixed-model sessions classify by.
`user`	The end user: attribution, and per-user limits and budgets in Directives.
`tags`	Up to ten tags for cost breakdowns in Usage and Logs.
`suggestion`, `decision`, `confirmedBy`	Accept a suggested model, or decline it with `decision: "decline"`. See below.
`task`	The turn's task class (`coding`, `extraction`, …), when your app knows it. The classifier is skipped.

Anything else in providerOptions.inferencebureau goes to the model's own provider, such as reasoningEffort on chat.

Model suggestions

With mixed-model sessions on for the key, each turn may come back with a cheaper model that meets your quality floor. Nothing switches until you send the suggestion back on the next turn:

let pending: string | null = null;

async function turn(messages: ModelMessage[]) {
  const result = streamText({
    // Always the model you'd normally use: an accepted suggestion swaps it on our side.
    model: ib("anthropic/claude-sonnet-5"),
    messages,
    providerOptions: { inferencebureau: { session: conversationId, ...(pending ? { suggestion: pending } : {}) } },
  });
  // At the end of the stream: a cheaper model for the next turn, if the classifier found one.
  pending = (await result.providerMetadata)?.inferencebureau?.suggestion as string | null ?? null;
  return result;
}

What comes back

providerMetadata.inferencebureau has what Inference Bureau said about the request, on generateText and at the end of streamText. inferenceBureauMetadata(headers) reads the same fields from any response's headers.

const result = await generateText({ model: ib("openai/gpt-5"), prompt });
const meta = result.providerMetadata?.inferencebureau;
// {
//   requestId: "req_…",            the receipt: GET /v1/requests/req_…, and Logs
//   session: "conv_42",
//   suggestion: "sug_…" | null,    suggestedModel, taskClass, confidence, suggestionReason
//   suggestionStatus: "accepted",  what happened to the suggestion this turn sent
//   pinned: "tool_cycle" | null,   a tool loop is open: the model stays put
//   fallbackModel: null,           the model that answered when model fallback was used
// }

Errors and limits

Refusals arrive as the AI SDK's APICallError, with the status, the error message and the request id. A request over a limit, or one your balance can't cover, is refused before it reaches a provider and isn't charged.

import { APICallError } from "ai";

try {
  await generateText({ model: ib("openai/gpt-5"), prompt });
} catch (e) {
  if (APICallError.isInstance(e)) {
    // 402 insufficient_balance, 429 rate_limit_exceeded, 403 when Directives refuse the model, …
    console.error(e.statusCode, e.responseHeaders?.["x-request-id"], e.message);
  }
}

Each request reserves its input plus its output budget against your team's tokens per minute, 2,000,000 by default. The budget is maxOutputTokens; without it, ib(id) reserves 4,096 tokens, but ib.messages(id) reserves the model's whole output limit (128,000 on Claude Sonnet 5), so set maxOutputTokens there. Agents that run many long steps at once can still reach it; raise limits.tokensPerMinute in Directives, and set a budget there.

Without the package

@ai-sdk/openai-compatible also works on its own, pointed at the Inference Bureau base URL. Send Inference Bureau's options as headers (ib-session, ib-user, ib-tags) and read its response headers yourself.

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { streamText } from "ai";

const ib = createOpenAICompatible({
  name: "inferencebureau",
  baseURL: "https://api.inferencebureau.com/v1",
  apiKey: process.env.IB_API_KEY,
  includeUsage: true,
  supportsStructuredOutputs: true,
});

const result = streamText({
  model: ib("anthropic/claude-sonnet-5"),
  messages,
  headers: { "ib-session": conversationId, "ib-user": userId },
});