API reference
Evaluations
Dashboard
BetaWe just launched Apps: hosted models, actions and pages, on the same bill as your inference.Read the Apps docs

Create an evaluation

POSThttps://api.inferencebureau.com/v1/evaluate

Classify or score any JSON with TypeSafe Jev. Billed like chat, with a receipt in Logs.

Body

application/json, up to 1 MiB; unknown fields refused
  • modelstringrequired

    Only typesafe-ai/jev. A chat model is refused with 400 invalid_request.

  • stateany JSONrequired

    What to evaluate: a conversation, a document, a record. Sent to the model as data.

  • questionsobjectrequired

    Map of your question ids (up to 64 characters) to questions, at least one. Each has a type and instructions (up to 4,000 characters): choice takes criteria as an object of option to description (two or more); score takes criteria as an array of level descriptions (two or more); boolean takes optional criteria with true and false.

  • providerOptionsobjectoptional

    Accepted for compatibility and ignored. Dispatch always uses certified providers.

  • userstringoptional

    Your end user, recorded for attribution and per-end-user Directives, and not sent to the model.

Response

  • answersobjectrequired

    One answer per question id. choice: choice, probabilities, confidence. score: score, probabilities, confidence. boolean: probability.

  • usageobjectrequired

    inputTokens and outputTokens. Jev is billed on input tokens only.

Headers

  • Authorizationstringrequired

    Bearer ib_live_…, or the key in x-api-key. See Authentication.

  • X-Client-Request-IDstringoptional

    Your own correlation id, recorded on the receipt.

  • ib-sessionstringoptional

    Groups related evaluations in Logs (session = …), such as one batch.

  • ib-userstringoptional

    Your end user, such as ib-user-1042, up to 256 characters; wins over user. See End users and tags.

  • ib-tagsstringoptional

    Comma-separated tags for cost breakdowns and tag budgets, at most ten. Never sent upstream.

Data handling

If your Directives require zero data retention, calls go only to Jev's ZDR provider (TypeSafe), with retention disabled. If your model allowlist leaves Jev out, the call returns 403 policy_denied before any content is sent, and Jev is not listed by GET /v1/models.

Limits

Your Directives' rate limits and budgets apply as on chat, including per-end-user limits and budgets for the end user a call names and tag budgets for its tags; endUser.required refuses a call that names no end user.

Request bodies are limited to 1 MiB. A call has 10 seconds. When a provider is at capacity the call is retried once inside that window, on Jev's other provider when your Directives allow it; a second 429 returns 429 upstream_rate_limited and nothing is charged.