Chat completions

POST /v1/chat/completions follows OpenAI’s shape. Zabio stores nothing from these calls unless your API key has body logging on (see Teams). The key needs the chat scope (keys have it by default).

For the exact shape of every request and response, with curl, Python and Node examples, see the API reference.

Request fields

Field	Supported	Notes
model	no	a catalog id or auto, auto/best, auto/balanced, auto/cheap. Omitted means auto
messages	yes	1 to 4096 messages. Roles: system, developer, user, assistant, tool. system and developer messages are joined into one system prompt. The legacy function role is refused
messages[].content	yes	a string, or an array of parts. user messages accept text and image_url parts; other roles accept text parts only
tools	yes	up to 256 tools of type function. parameters defaults to an empty object schema
tool_choice	yes	auto, none, required or {"type": "function", "function": {"name": "..."}}; needs tools unless it is none
parallel_tool_calls	yes	passed to the vendor where supported
max_tokens, max_completion_tokens	yes	max_completion_tokens wins when both are present. Default 16000. A value above the served model’s maximum output is lowered to that limit, on every attempt, so auto and failover never send a vendor a budget it rejects
temperature	yes	0 to 2. Left out for a model that does not take it, and named in zabio.dropped_parameters and the x-zabio-dropped-params header; see Sampling parameters by model
top_p	yes	0 to 1. Left out for a model that does not take it, and named in zabio.dropped_parameters and the x-zabio-dropped-params header; see Sampling parameters by model
stop	yes	a string or up to 4 strings. Where the model’s provider does not take it, Zabio cuts the answer at the first sequence itself, so it is always honored
stream	yes	server sent events, see below
stream_options.include_usage	accepted	the final usage chunk is always sent, so this flag changes nothing
response_format	yes	text, json_object or json_schema; see structured output below
reasoning_effort	yes	none, minimal, low, medium, high or xhigh, one scale for every vendor, mapped onto the served model’s own control and reported in zabio.reasoning. On auto, leave it out and Zabio chooses the setting with the model; see Reasoning levels
user, metadata	yes	forwarded as request metadata
n	only 1	n greater than 1 is refused
seed, logit_bias, presence_penalty, frequency_penalty	ignored	accepted so SDK defaults do not break; listed in zabio.ignored_parameters when they carry a non default value
logprobs, top_logprobs, audio, prediction, web_search_options, non text modalities	refused	400 unsupported_parameter
zabio	yes	Zabio’s options, below

Request bodies are limited to 32 MB (413 payload_too_large).

The zabio request object

{
  "model": "auto",
  "messages": [...],
  "zabio": {
    "session_id": "chat-7f3a",
    "vendors": ["anthropic", "openai"],
    "fallback": "same_vendor",
    "cache": {"ttl": "1h", "breakpoints": "auto"},
    "split": "auto",
    "deadline": "30m"
  }
}

Field	Values	What it does
session_id	any string up to 128 characters	groups calls so the routing decision is reused and signals accumulate. See Models and routing
vendors	list of anthropic, openai, google	only these vendors are candidates
fallback	same_vendor, cross_vendor, none	what happens when an explicitly named model is unavailable. Default: the key’s fallback_policy
cache.ttl	5m, 1h, none	prompt cache lifetime where the vendor offers one. Anthropic: 5 minutes or 1 hour (1 hour costs more to write). OpenAI caching is automatic and this value is ignored. Google, and the other vendors with a published cache price, cache automatically, so this value is ignored for them too. Caching is automatic on every call, to auto or to a named model; none turns it off. Default: 5m, the lifetime used when Zabio writes. A long prompt (1,024 tokens or more) is written to the cache on its second appearance, not its first: the first time it is sent without a write, and when the same prompt or the same system prompt and tools come back within the lifetime it is written, and later calls read it. A prompt sent once never pays the write. Calls in a session_id, requests that carry earlier turns of the conversation, and requests that set cache.ttl themselves are written at once. With that default, a call in a session_id whose previous turn was more than five minutes ago gets 1h instead: a 5 minute cache would have expired and be rewritten on every turn, which costs more than no cache. zabio.cache.ttl_applied reports the choice. Inside a batch the default is none: a batch line caches only when it asks
cache.breakpoints	auto, none	auto places cache markers on the system prompt, the last tool, the last message and the previous user turn, so a conversation’s history is cached turn to turn; none disables caching for this call and forces ttl to none
split	auto, off	whether Zabio may split a big request under auto (see Split and route): a cheap model copies the relevant lines of a large source and the routed model answers on that excerpt, when that is cheaper. off sends the request whole. Default auto. Not a key setting: the choice lives in the request
deadline	a duration with units s, m or h (15m, 1h, 24h), up to 168h	how long the answer may take. Absent or under 15 minutes: nothing changes. From 15 minutes up, the call may be served on the vendor’s flex tier where the model has one: the same model at about half the price, slower (see Flex tier). Not a guarantee of a delivery time, and not a key setting: the choice lives in the request

If your SDK rejects unknown body fields, send the same values as headers. Headers win over the body.

Header	Same as
x-zabio-session-id	zabio.session_id
x-zabio-cache-ttl	zabio.cache.ttl
x-zabio-fallback	zabio.fallback
x-zabio-split	zabio.split
x-zabio-deadline	zabio.deadline

Streaming

With stream: true the response is text/event-stream. Each event is a data: line with a chat.completion.chunk. The order is: a chunk with delta.role, content and tool call deltas, a chunk with finish_reason, then one usage chunk with empty choices, the usage block and the zabio object, then data: [DONE].

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":9,"total_tokens":21},"zabio":{...}}

data: [DONE]

The usage chunk is always sent. If the vendor connection fails mid stream, an error object is sent in a chunk and the stream ends.

Long calls

Some calls take minutes: a long output, or a powerful model that thinks for a while before it answers. Zabio allows one vendor attempt up to 600 seconds on powerful and frontier models (180 seconds on the others) and keeps the connection alive, so nothing between you and the vendor gives up first.

• Not streamed. If the vendor has not answered after 90 seconds, Zabio commits the 200 status and headers and writes one space every 20 seconds until the answer arrives; the JSON body follows the spaces, and every JSON parser skips them. A failure after that point cannot change the status any more, so the body is then a chat.completion with an empty choices list and an error object beside it (message, type, code). Check error when choices is empty. Calls answered within 90 seconds are exactly what they always were, status codes included.
• Streamed. From about 20 seconds in, and whenever the vendor is silent for 20 seconds, Zabio sends an SSE comment line (: keepalive), which SSE clients ignore. A failure before any data arrives comes as an error event, data: {"error": {...}}, followed by data: [DONE]; the official SDKs raise on it.
• Your own timeout. Set your client’s request timeout to at least 600 seconds for powerful and frontier models. The OpenAI SDKs default to 600 seconds; LangChain and the Vercel AI SDK have no default timeout.
• If you disconnect. Closing the connection cancels the vendor call. A call that was not streamed costs you nothing; a stream is billed for the tokens delivered before the disconnect.
• Headers on a kept-alive call. The x-zabio-* headers were sent before the vendor was chosen, so on such a call they carry no vendor; the vendor, the model served and the routing are in the body’s zabio object, as always.

Split and route

Under auto, a big request can be answered in two steps when that is cheaper: a cheap model copies, word for word, the lines of your source that the question needs, and the model routing chose answers on that excerpt instead of the whole source. Zabio does this only when the last user message is plain text of several thousand tokens with a separate question that needs part of it, the request carries no tools or images, its own estimate says the split costs at most 70 percent of the unsplit call, a check confirms the excerpt holds what the question names, and every guard passes; otherwise the request goes whole, as always. The answering model is told the material was excerpted and can reply with the line NEED_MORE_SOURCE, in which case Zabio sends the full request to the same model and returns that answer. A named model is never split. Send zabio.split: "off" (or the header x-zabio-split: off) to send a request whole. Every step, model and cost is in zabio.split, and a split never costs you more than the unsplit call. See Split and route for what the model sees and Billing for the fee.

Classification jobs on auto

Under auto, a request whose response_format is a json_schema made only of fixed choices (enum), booleans and bounded integers or numbers, with no tools, images or stream, may be answered by Jev (jev-1.13.0) instead of the text model routing chose, when Zabio’s estimate says Jev is cheaper. Jev answers in about 0.15 seconds with the calibrated probability of every answer; when any field of its answer is under 0.5, Zabio discards it and the text model answers as always, at Zabio’s cost. You can tell from zabio.model_served (jev-1.13.0) and zabio.decision; the answer keeps your schema’s shape. A named model is never answered by Jev. See Classification jobs on auto.

Flex tier

OpenAI (flex processing) and Google (Flex inference) serve the same model at about half the price when you can wait: their own latency target is up to 15 minutes, and capacity is best effort. Zabio uses these tiers when your request says it can wait: send zabio.deadline (or the header x-zabio-deadline) with a duration of 15 minutes or more, for example "30m" or "2h". A deadline under 15 minutes, or none, changes nothing.

• Which models. OpenAI’s GPT-5, GPT-5.x and GPT-6 chat models, o3 and o4-mini; every Google Gemini model. Flex prices are half the standard input, output and cache read prices, except the cached input of gpt-5.4 and o4-mini, which OpenAI lists at 0.13 and 0.138 USD per million tokens. Every other model (the GPT-4 family, o1, o3-mini, chat-latest, every Anthropic, DeepSeek, xAI, Mistral and Together AI model) has no flex tier and is served and priced at standard, deadline or not.
• With auto. Routing compares the candidates at their flex prices where a model has one, so auto can pick a model because it is cheaper in flex. A named model is served at flex when it has a flex price.
• Never a failed call. When the vendor refuses a flex request for capacity (OpenAI answers 429 resource unavailable, Google 503 or 429), Zabio retries the same model at the standard tier at once and bills the price actually used. zabio.tier.fallback is then capacity.
• Time. A flex attempt may take up to 900 seconds; Zabio keeps the connection alive under it as described in Long calls. Set your client’s timeout to at least 900 seconds when you send a deadline.
• What you see. zabio.tier reports requested (flex or standard, what your deadline mapped to), applied (what served the call) and fallback (capacity when flex was refused).
• The fee. The saving of a flex call is measured against the flex price of your key’s baseline model, or its standard price when the baseline model has no flex tier, since that is what you would have paid there. Zabio never takes a cut of the vendor’s own discount. See Billing.
• Split and route. The pre-steps of a split (the extract and the checks) run at the standard tier; only the answer can be flex.

Work that can wait a day goes through Batches (/v1/files and /v1/batches), where every line is priced at the vendors’ batch rates, or through a background response with a deadline of 24 hours or more, which joins your organization’s next batch without a file. A synchronous call with such a deadline cannot wait that long on one connection, so it is served at the flex tier where the model has one (at standard otherwise) and zabio.tier says so: {"requested": "batch", "applied": "flex", "fallback": "synchronous"}.

Images

user messages may contain image_url parts:

{"role": "user", "content": [
  {"type": "text", "text": "What is in this picture?"},
  {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}}
]}

url is either an http(s) URL or a data URI of the form data:image/<type>;base64,<data>. Zabio downloads URLs and sends the bytes to the vendor, so every vendor gets the same input. Rules for URLs:

• public hosts only; addresses in private, loopback, link local or otherwise internal ranges are refused
• at most 20 MB per image, up to 20 images per request
• the server must answer with image/png, image/jpeg, image/webp or image/gif
• at most 3 redirects, http(s) only, 15 second timeout

Failures return 400 invalid_image_url with param pointing at the failing part. A request with images is only routed to vision capable models.

Tools

Function tools work as in OpenAI. Send tools, receive tool_calls on the assistant message with finish_reason: "tool_calls", answer with tool messages carrying tool_call_id. Arguments are always a JSON string ({} when empty). Requests with tools are only routed to tool capable models.

Structured output

response_format.type	Behaviour
text	default
json_object	OpenAI, Google, DeepSeek, xAI, Mistral AI and Together AI use their native JSON mode. Anthropic has no JSON object mode: Zabio adds a system instruction asking for exactly one JSON object and nothing else, starts the reply inside the object on models that allow it, and strips a Markdown code fence around the object when the model adds one anyway. This is best effort by construction. Validate the output on your side when the answer may come from an Anthropic model, and prefer json_schema whenever you know the shape you want
json_schema	{"type": "json_schema", "json_schema": {"name": "...", "schema": {...}, "strict": true}}. Mapped to each vendor’s native structured output. schema is required. Only routed to models with json_schema support (not the DeepSeek direct models). On jev-1.13.0, the decision model, the schema’s properties are the questions Jev answers, each with its probability in zabio.decision; see Decision models

The response

A standard chat.completion object. usage.prompt_tokens counts every input token, cached or not, so totals match what the vendor processed; usage.prompt_tokens_details.cached_tokens is the cached part, and usage.completion_tokens_details.reasoning_tokens is the vendor’s count of thinking tokens inside completion_tokens, present when the vendor reports one. finish_reason is stop, length, tool_calls or content_filter. Zabio serves exactly one choice. A reasoning model that spends its whole output budget thinking returns an empty answer with finish_reason: "length": raise max_tokens so it leaves room for the answer above the thinking, or lower reasoning_effort.

The zabio response object

Field	Type	Meaning
vendor	string	who served the call: anthropic, openai, google, deepseek, xai, mistral, together or typesafe
model_served	string	the model that answered; differs from model you sent for auto and fallbacks
routing	object or absent	line, task, task_type (the kind of work as a tag, see Kinds of work), difficulty, band, stakes, biased, cell, confidence (0 to 1), reason (the classifier’s sentence), trigger (why the decision was made; see Models and routing), classifier (who decided: typesafe/jev-1.13.0, or the text classifier it fell back to; absent on explicit models), effort (the reasoning setting chosen with the model), effort_reason (why) and sticky (present only when the session’s warm prompt cache decided the pick: the model kept and the one cold pricing would have chosen). Absent on explicit models without routing information
reasoning	object	requested (the level you sent, omitted when none) and applied (what the served model got: default, off, the vendor’s level name, budget N, on, always on or not supported)
cost_micros	integer	what you were billed, in millionths of a US dollar (1,000,000 micros = 1 USD)
cost	object	the breakdown: vendor_micros (provider list price), service_fee_micros, call_fee_micros (the fixed fee an auto request pays once, the price of automatic model choice; 0 on a named model), savings_fee_micros and classifier_micros (both always 0, kept so existing clients keep parsing them), direct_micros (the same call on your key’s baseline model with the same cache reads and writes; equals the provider price on a named model) and saved_micros (direct_micros minus cost_micros; can be negative; left out on auto while your key’s baseline is pending); on auto, baseline says whether your key has a baseline yet (pending, model) or how many of the 20 successful calls it needs it has (calls, calls_needed)
byok	boolean	true when the call used your own vendor key
cache	object	read_tokens, write_tokens and ttl_applied (5m, 1h or empty)
session_id	string	echoes the session id you sent
escalated_from	string	the line and model whose answer auto replaced: on the Responses API when that model asked for a stronger one, and on any request that is not streamed when its answer failed Zabio’s check
compacted	boolean	Responses API only: the stored history was summarized before this turn
vendor_stop_reason	string	the vendor’s own stop reason, for debugging
ignored_parameters	list	request fields Zabio accepted and ignored
dropped_parameters	list	sampling parameters the model that answered does not take (temperature, top_p), left out of its request; also the x-zabio-dropped-params header
attempts	integer	how many providers were tried before one answered
decision	object	when jev-1.13.0 answered, named or chosen by auto for a classification job: the calibrated probabilities behind each property of the answer, see Decision models
split	object	present while split and route is on: applied (true when the answer came from an excerpt of your source), pattern (distill), reason (why the request was not split: named_model, opt_out, not_eligible, below_threshold, jev_no, jev_failed, price_check), estimate (unsplit_micros, split_micros), steps (every call of the split: role questions, extract, check or answer, with vendor, model, input_tokens, output_tokens, micros at list price and billed), excerpt (lines_kept, lines_total, tokens, verbatim_rate), check_p (the check’s probability), fallback (the guard that sent the request whole) and saved_micros (on a split that held, what the split saved against the request sent whole, at list price; shown while your key’s baseline is pending too). See Split and route
tier	object	the service tier: requested (standard, flex or batch, what your deadline mapped to: flex from 15 minutes, batch from 24 hours), applied (what served the call) and fallback (capacity when flex was requested and the vendor refused it, so standard served and was billed; synchronous when the deadline asked for batch on a synchronous call, which cannot wait a day, so flex or standard served). See Flex tier
plan	object	present on a call served under the Free limits (see Billing): tier (free), reason (free_plan, the organization never topped up; zero_balance, a paid organization at zero continuing on the free tier), paid_would_use (on auto, the model a paid account would have been given, when it differs from the one that answered) and topup_url. Absent on a paid call. Not to be confused with tier, the vendor processing tier

Fields with empty values are omitted, so check for presence.

Response headers

Header	Meaning
x-request-id	id to quote in support requests
x-zabio-vendor	same as zabio.vendor. On a call kept alive past 90 seconds (see Long calls) the headers went out before the vendor was chosen, so read vendor, model and routing from the body’s zabio object
x-zabio-byok	true when your own vendor key was used
x-zabio-balance-micros	your organization’s credit balance after the admission check
x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests	the key’s requests per minute bucket
x-ratelimit-limit-tokens, x-ratelimit-remaining-tokens, x-ratelimit-reset-tokens	the key’s tokens per minute bucket. It is charged up front with the input estimate plus the output budget, clamped to what the served model can produce, and settled after the call; a max_tokens far above what the job needs can hit the limit early, so set it near what you need
x-zabio-quota-scope, x-zabio-quota-limit, x-zabio-quota-remaining	present when a spend limit applies to the key (key, over its reset period) or the monthly quota of the member who created it (member)
x-zabio-plan	free on every call served under the Free limits; absent on a paid call
x-ratelimit-limit-requests-day, x-ratelimit-remaining-requests-day, x-ratelimit-reset-requests-day	the Free plan’s day: 100 successful requests, what is left, and the time to 00:00 UTC. Only with x-zabio-plan: free; on such calls x-ratelimit-*-requests is the Free minute bucket (20), which is tighter than the key’s own
x-zabio-free-limit	on a 429 or 403 free_tier_limit, which limit was met: requests_per_day, requests_per_minute, daily_cost, input_tokens, max_tokens, members or api_keys
Retry-After	seconds to wait, on 429 and on retryable 5xx

See Errors for what happens when a limit is hit.