Responses API and conversations

POST /v1/responses implements the OpenAI Responses API shape, so the OpenAI SDKs work unchanged. Unlike chat completions, which store nothing, a response is stored on Zabio’s side together with the conversation it belongs to. You send the new message; Zabio assembles the history, routes it, stores the answer and returns it.

For the exact shape of every request and response, with curl, Python and Node examples, see the API reference.

The key needs the conversations scope (keys have it by default). A key without it receives 403 insufficient_scope.

Create a response

from openai import OpenAI

client = OpenAI(base_url="https://api.zabio.ai/v1", api_key="insert your zabio.ai key here")

first = client.responses.create(model="auto", input="My name is Ana. Remember it.")
second = client.responses.create(model="auto", input="What is my name?", previous_response_id=first.id)
print(second.output_text)

Field	Type	Notes
`model`	string	a catalog id or `auto`, `auto/best`, `auto/balanced`, `auto/cheap`. Omitted means `auto`
`input`	string or list of items	a string is one user message. Items: `{"role": "user" or "assistant" or "system" or "developer", "content": string or parts}`, `{"type": "function_call_output", "call_id", "output"}`, `{"type": "function_call", "call_id", "name", "arguments"}`. Content parts: `input_text`, `input_image` (`image_url`)
`instructions`	string	system instructions for this turn (stored with the turn)
`previous_response_id`	`resp_...`	continue from that response. Pointing at an older response branches: the conversation keeps its own thread and this response starts a new one
`conversation`	`conv_...` or `{"id": ...}`	continue from the conversation’s latest turn. With `previous_response_id`, both must belong to the same conversation
`store`	bool, default true	`false` computes the turn from the stored history and stores nothing. The response id is not retrievable afterwards
`tools`, `tool_choice`		function tools: `{"type": "function", "name", "description", "parameters"}`; `tool_choice` is `auto`, `none`, `required` or `{"type": "function", "name"}`
`max_output_tokens`, `temperature`, `top_p`, `parallel_tool_calls`		sampling; `max_output_tokens` defaults to 4096. A value above the served model’s maximum output is lowered to that limit, on every attempt
`text.format`		`{"type": "json_object"}` or `{"type": "json_schema", "name", "schema", "strict"}`. The Anthropic `json_object` caveat from Chat completions applies
`reasoning.effort`		`none`, `minimal`, `low`, `medium`, `high` or `xhigh`, mapped onto the served model’s own control; `zabio.reasoning` reports what was applied
`metadata`	object	up to 16 keys, 64 character keys, 512 character values
`stream`	bool	server sent events, see below
`background`	bool, default false	run the turn later and return at once with `status: "queued"`; poll `GET /v1/responses/{id}` for the result. Needs `store` true and no `stream`. See Background responses
`zabio`	object	`session_id`, `vendors`, `fallback`, `cache`, `split`, `deadline` as in chat completions, plus `retention_days` (0 to 365, for a conversation created by this call) and `compaction` (`auto` or `off`)

Without previous_response_id or conversation a new conversation is created for the response when store is true.

The response object

{
  "id": "resp_...", "object": "response", "created_at": 1758283200,
  "status": "completed",
  "model": "claude-haiku-4-5",
  "output": [
    {"type": "message", "id": "msg_...", "status": "completed", "role": "assistant",
     "content": [{"type": "output_text", "text": "...", "annotations": []}]},
    {"type": "function_call", "id": "fc_...", "call_id": "call_...", "name": "get_weather", "arguments": "{\"city\":\"Lima\"}", "status": "completed"}
  ],
  "usage": {"input_tokens": 120, "input_tokens_details": {"cached_tokens": 100}, "output_tokens": 20, "output_tokens_details": {"reasoning_tokens": 0}, "total_tokens": 140},
  "previous_response_id": null,
  "conversation": {"id": "conv_..."},
  "metadata": {}, "store": true,
  "zabio": {"vendor": "anthropic", "model_served": "claude-haiku-4-5", "routing": {...}, "cost_micros": 63, "cost": {...}, "byok": false, "escalated_from": "", "compacted": false, "split": {...}}
}

status is completed, incomplete (the output hit max_output_tokens; incomplete_details.reason says so; a reasoning model that spent its whole budget thinking ends this way with no text, so raise max_output_tokens so it leaves room for the answer above the thinking, or lower the reasoning effort) or failed; a background response is also queued or cancelled, and carries background: true. usage.output_tokens_details.reasoning_tokens is the vendor’s thinking count when it reports one, else 0. The zabio object is the one described in Chat completions, and here three more fields matter:

escalated_from: set when the first model chosen by auto asked for a stronger one, or when its answer failed Zabio’s check, and the answer came from another model. The escalation itself never appears in output.
compacted: set when the stored history was summarized before this turn because it no longer fit the model’s context window.
split: what split and route did to this turn (see Split and route), stored with the response like compacted. Only the turn’s own last user message can be split, never the stored history; zabio.split: "off" sends the turn whole.
tier: the service tier of this turn (see Flex tier): requested, applied and fallback, stored with the response like split. A zabio.deadline of 15 minutes or more makes the turn eligible for the vendor’s flex tier; one of 24 hours or more asks for the batch tier, which only a background response can wait for. On a synchronous turn with such a deadline applied is flex or standard and fallback is synchronous.
plan: present on a turn served under the Free limits (see Billing, “The Free plan”): tier (free), reason (free_plan or zero_balance), paid_would_use (on auto, the model a paid account would have been given, when it differs) and topup_url. A background response carries it on the queued object and on the final one; a turn admitted with credit is billed paid to its end even if the balance crosses zero while it runs.

Streaming

With stream: true the response is a stream of server sent events: response.created, response.in_progress, response.output_item.added, response.content_part.added, response.output_text.delta, response.output_text.done, response.content_part.done, response.output_item.done, response.function_call_arguments.delta, response.function_call_arguments.done, and finally response.completed, response.incomplete or response.failed with the full response object. Each event carries sequence_number. On a vendor failure an error event is sent.

const stream = await client.responses.create({ model: "auto", input: "Hello", stream: true });
for await (const event of stream) {
  if (event.type === "response.output_text.delta") process.stdout.write(event.delta);
  if (event.type === "response.completed") console.log("\n", event.response.conversation.id, event.response.zabio);
}

Background responses

Send background: true and the call returns at once with the response object in status: "queued", background: true and an empty output. The turn runs later; poll GET /v1/responses/{id} until the status is completed, incomplete, failed or cancelled, then read output, usage and zabio as on any response. Everything else about the turn is the same: the conversation, previous_response_id, tools, compaction, the zabio options. background needs store true (the default; the response is stored so it can be polled) and cannot be combined with stream.

Where the turn runs depends on zabio.deadline:

A deadline of 24 hours or more on auto or on a model with a batch tier puts the turn in your organization’s next batch: single background requests are gathered for up to 15 minutes, or until 500 have arrived, and sent together as one vendor batch per model at the vendors’ batch prices, about half of the list price. The result arrives within the vendors’ 24-hour window from that moment, usually much sooner. The turn is billed exactly like a line of a batch: the batch price for the tokens it used, plus the usual fees, with the saving measured against your baseline model at its batch price. zabio.tier is {"requested": "batch", "applied": "batch"}. Before the turn is queued, its estimated cost is set aside on your balance like a batch’s; if the estimate exceeds your available balance the call answers 402 insufficient_credits and nothing is stored.
Any other background request runs as soon as a worker picks it up, through the same routing as a synchronous call: at the flex tier from a deadline of 15 minutes where the model has one, at standard otherwise. A deadline of 24 hours or more on a model whose vendor has no batch tier runs this way too, and says so: zabio.tier is {"requested": "batch", "applied": "flex" (or "standard"), "fallback": "no_batch_tier"}. Nothing is silently moved to a dearer tier.

The queued object already carries zabio.tier with the tier the request got; the final object carries what served it.

Cancel. POST /v1/responses/{id}/cancel ends a queued turn, or one still waiting for its batch to be sent, with status: "cancelled" and nothing billed; its inputs stay in the conversation with that status, like a failed turn’s. A turn already running as a job is cut and ends cancelled. A turn already sent to the model vendor inside a batch cannot be pulled back: the call answers 409 not_cancellable and the answer arrives and is billed. Cancelling a turn that has ended returns the object as it is; cancelling a response that was not created with background answers 400 not_background.

Python

r = client.responses.create(model="auto", input="Summarize this report: ...", background=True,
                            extra_body={"zabio": {"deadline": "24h"}})
print(r.status)                      # queued
while r.status in ("queued", "in_progress"):
    time.sleep(60)
    r = client.responses.retrieve(r.id)
print(r.status, r.output_text, r.model_extra["zabio"]["tier"])

Node

let r = await client.responses.create({ model: "auto", input: "Summarize this report: ...", background: true,
                                        zabio: { deadline: "24h" } });
while (r.status === "queued" || r.status === "in_progress") {
  await new Promise((ok) => setTimeout(ok, 60_000));
  r = await client.responses.retrieve(r.id);
}
console.log(r.status, r.output_text, r.zabio.tier);

Errors specific to this endpoint

Status	code	When
400	`invalid_value` on `background`	`background` with `stream`, or with `store: false`
400	`background_unavailable`	background responses are not available on this server
400	`not_background`	cancel on a response that was not created with `background`
409	`not_cancellable`	cancel on a turn already sent to the model vendor inside a batch; the answer arrives and is billed
400	`conversation_mismatch`	`previous_response_id` belongs to another conversation
400	`conversation_too_large`	the conversation reached 4,000 entries or 2,000,000 estimated tokens on its thread; start a new one
400	`context_length_exceeded`	the history plus this turn does not fit the model’s context window and compaction is off or cannot help; the message carries the numbers
403	`conversations_closed`	stored conversations are closed for this organization
404	`previous_response_not_found`, `conversation_not_found`, `response_not_found`	unknown or deleted
503	`conversations_unavailable`	stored conversations are not available on this server

Everything in Errors applies as well, including 402 insufficient_credits and the free_tier_* codes of the Free plan.

Read and delete responses

GET /v1/responses/{id} returns a stored response with its output, or its current status when it is a background response still running.
POST /v1/responses/{id}/cancel cancels a background response.
DELETE /v1/responses/{id} removes the response with its turn (the inputs it stored and the outputs) and everything stored after it in the same thread. The conversation continues from where that turn started. Deleting a queued background response drops it from the queue.

Conversations

POST /v1/conversations with {"metadata": {...}, "items": [...], "zabio": {"retention_days": 30}} creates an empty (or seeded) conversation.
GET /v1/conversations/{id} returns {"id", "object": "conversation", "created_at", "metadata", "zabio": {"entries", "token_estimate", "expires_at", "last_turn_at", "retention_days"}}.
POST /v1/conversations/{id} with {"metadata": {...}} replaces the metadata.
DELETE /v1/conversations/{id} deletes the conversation and everything in it, immediately and permanently.
GET /v1/conversations/{id}/items?limit=20&after=msg_...&order=asc lists every stored item in creation order (all threads), including compaction summaries as {"type": "compaction", "summary": ...}.
POST /v1/conversations/{id}/items with {"items": [...]} appends items without a model turn, for example to seed history.

Which key sees what

A key that belongs to a project reads, continues, changes and deletes only the responses and conversations made with that project’s keys; a key at the organization’s level reaches the whole organization. A response or conversation outside answers 404 not_found, as if it did not exist. The project is the one the key that made it is in now. See Organizations, members and keys.

Escalation, stickiness and compaction

On auto turns the model may signal that the task exceeds its capability. Zabio then re-runs the same turn on the next model up the curve, one its knowledge says is probably better, with the full history and returns that answer; later turns stay on the stronger model. Inside a conversation auto classifies each new user message and reuses that job across the tool-result turns that follow, until a signal (a run of failed tool calls, a long pause, a low confidence classification, user pushback) triggers a new choice.

When the stored history plus a new turn would not fit the context window of the model the request is served by, Zabio summarizes the oldest part of the history with a fast model, stores the summary as a compaction item, and continues. The summary call runs on Zabio’s key, is billed like any other call and appears in the conversation’s responses; a turn that needs compaction therefore needs credit even when it names a model served on your own key, and at zero balance it runs under the Free terms like any auto call, or gets 402 insufficient_credits before any summary is made when the organization is set to stop. zabio.compaction: "off" disables it; the request then fails with context_length_exceeded and the numbers.

Long calls

A powerful model can think for minutes before its first token. Zabio allows one vendor attempt up to 600 seconds on powerful and frontier models (180 seconds on the others) and keeps the connection alive meanwhile.

Not streamed. After 90 seconds without an answer, Zabio commits the 200 status and headers and writes one space every 20 seconds until the response object arrives; JSON parsers skip the spaces. A failure after that point is delivered as the response object with status: "failed" and an error object (code, message), the shape a failed response has in any case. Calls answered within 90 seconds are unchanged, status codes included.
Streamed. From about 20 seconds in, before response.created if the vendor has not opened yet, and whenever the vendor is silent for 20 seconds, Zabio sends an SSE comment line (: keepalive), which SSE clients ignore. A failure before any output comes as a response.failed event carrying the object with status: "failed" and its error.
Your own timeout. Set your client’s request timeout to at least 600 seconds for powerful and frontier models.
If you disconnect. Closing the connection cancels the vendor call. A call that was not streamed costs you nothing; a stream is billed for the tokens delivered before the disconnect. A stored turn that was cut is marked failed or incomplete.
Headers on a kept-alive call. The x-zabio-* headers were sent before the vendor was chosen; the vendor, the model served and the routing are in the object’s zabio field.

Retention and deletion

Each organization has a default retention (30 days, adjustable from 1 to 365 in Settings). A conversation may set a shorter or longer retention_days (0 to 365) when it is created; 0 deletes it as soon as the response is returned. The retention clock restarts on every turn. Expired conversations are deleted nightly. Deleting through the API is immediate and permanent. Bodies are encrypted at rest with a key that belongs to your organization; closing the organization deletes the conversations and destroys the key. See Privacy.

Body logging for stateless calls

/v1/chat/completions and /v1/embeddings store nothing unless the API key has body logging on (a key setting, off by default). With it on, the request and response of each call are stored encrypted for the organization’s retention period, and owners and admins can read them while signed in to the web app. The call id is the response id without its chatcmpl- prefix, written as call_....