Responses API

Use POST /v1/responses for text generation, streaming, conversations, and model tool use. The API follows the OpenAI Responses request and response shapes where documented here, with platform-specific conversation handling and gateway-hosted tools.

For a single prompt and answer, send a string in input. For multiple turns, either send the history as an input array or let the platform store it and continue with conversation_id.

Before you begin

You need:

• the API URL shown in the platform portal
• an API key with invoke access to a chat model
• the model ID from GET /v1/models

The raw HTTP examples use curl. Set these values once:

export API_BASE_URL="https://api.<your-domain>"
export API_KEY="cm_api_…"
export MODEL_ID="YOUR_MODEL_ID"

The Python examples use the OpenAI Python package and the same environment variables. Install the package with your project's usual Python dependency manager.

Generate a response

Send a non-streaming request when you want one JSON result:

curl -sS "${API_BASE_URL}/v1/responses" \
  -H "Authorization: Bearer ${API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<JSON
{
  "model": "${MODEL_ID}",
  "input": "Explain confidential computing in two sentences.",
  "store": false
}
JSON

The answer is in the output array. A message can contain one or more output_text parts:

{
  "status": "completed",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "Confidential computing protects data while it is being processed..."
        }
      ]
    }
  ]
}

With the OpenAI Python client:

import os

from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['API_BASE_URL']}/v1",
    api_key=os.environ["API_KEY"],
)

response = client.responses.create(
    model=os.environ["MODEL_ID"],
    input="Explain confidential computing in two sentences.",
    store=False,
)

if response.status != "completed":
    raise RuntimeError(f"Response ended with status {response.status}: {response.error}")

print(response.output_text)

For direct model requests, store: false makes this a one-shot request. Storage otherwise defaults to true, and the gateway creates or continues a conversation independently of the selected model backend.

Give the model instructions

Use instructions for behavior that should apply to the current request:

{
  "model": "YOUR_MODEL_ID",
  "instructions": "Answer for a non-technical reader. Be concise.",
  "input": "What is a trusted execution environment?",
  "store": false
}

For direct model requests, parameters such as max_output_tokens, temperature, and top_p can be sent when the selected model and its configuration support them. Preset agents use their configured parameters; callers cannot override them.

Continue a conversation

You can keep history in your application or let the platform store it.

Approach	When to use it
Send an input array	Portable client-managed pattern; your application owns and resends the history.
Use conversation_id	Platform-stored history; later requests send only the new input.

Preset-agent requests accept only user messages from the caller, so use platform-stored history instead of resending client-managed history. Storage is disabled by default: set store: true on the first turn, then continue with the returned conversation_id. Do not combine conversation_id with store: false; the request returns 400.

Send history yourself

Start with a user message, then add the response output and the next user message to the next request:

history = [{"role": "user", "content": "My deployment window starts at 09:00 UTC."}]

first = client.responses.create(
    model=os.environ["MODEL_ID"],
    input=history,
    store=False,
)

history.extend(item.model_dump() for item in first.output)
history.append({"role": "user", "content": "What time does my deployment window start?"})

second = client.responses.create(
    model=os.environ["MODEL_ID"],
    input=history,
    store=False,
)

print(second.output_text)

Set store: false on every client-managed turn. Storage is enabled by default, so omitting it would create a new stored conversation for each request even though your application is already supplying the history.

Let the platform store history

On the first request, omit conversation_id. The response contains a new ID in conversation.id:

curl -sS "${API_BASE_URL}/v1/responses" \
  -H "Authorization: Bearer ${API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<JSON
{
  "model": "${MODEL_ID}",
  "input": "My deployment window starts at 09:00 UTC. Remember that.",
  "store": true
}
JSON

Copy the ID from the response's conversation object:

{
  "conversation": {
    "id": "YOUR_CONVERSATION_ID"
  }
}

Pass that ID as conversation_id on the next request. You send only the new input; the gateway loads the stored history and supplies it to the model:

curl -sS "${API_BASE_URL}/v1/responses" \
  -H "Authorization: Bearer ${API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<JSON
{
  "model": "${MODEL_ID}",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "input": "What time does my deployment window start?"
}
JSON

The same flow with the OpenAI Python client uses extra_body because conversation_id is a platform extension:

first = client.responses.create(
    model=os.environ["MODEL_ID"],
    input="My deployment window starts at 09:00 UTC. Remember that.",
    store=True,
)

if first.conversation is None:
    raise RuntimeError("The gateway did not create a conversation")

conversation_id = first.conversation.id

second = client.responses.create(
    model=os.environ["MODEL_ID"],
    input="What time does my deployment window start?",
    extra_body={"conversation_id": conversation_id},
)

print(second.output_text)

Keep the conversation_id in your application's session or database if the conversation must survive a process restart. Start a new conversation by omitting it from a later request. Use the same API key for later turns.

Conversation behavior and limits

Behavior	What to expect
Create	Omit conversation_id and set store: true. The response contains conversation.id.
Continue	Send the stored ID as conversation_id; the gateway supplies recent history before the new input.
History window	Up to 50 recent stored items are supplied. The model's context limit still applies.
Access	A conversation is scoped to its creator and tenant. Keep using the same API key.
Disable storage	Set store: false. No conversation is created and conversation is absent from the response.
Change models	Prefer the same model. Model-specific reasoning and tool items may be omitted after a change.
Manage conversations	The public /v1 API currently does not expose list, retrieve, or delete operations.

conversation_id is the supported mechanism for continuing platform-stored history. The previous_response_id and conversation request fields are not currently supported and do not continue a platform conversation.

Stream a response

Set stream: true to receive Server-Sent Events. Text arrives in response.output_text.delta events, followed by a terminal response.completed, response.incomplete, or response.failed event and [DONE].

stream = client.responses.create(
    model=os.environ["MODEL_ID"],
    input="Write a four-line poem about private AI.",
    stream=True,
    store=False,
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
    elif event.type == "response.failed":
        raise RuntimeError(event.response.error)

print()

When storage is enabled, the terminal event's response.conversation.id contains the conversation ID to use for the next request.

Use tools

The Responses API can run function tools published by the Model Gateway, including RAG and connected MCP tools. The gateway can execute those tools and feed their results back to the model within the same request.

→ Use Model Gateway tools with the Responses API

Tool availability depends on the selected model's function-calling support and configuration. OpenAI-hosted tool types such as web search, file search, code interpreter, computer use, and image generation are not provided by this endpoint, regardless of the selected model.

tool_choice support depends on the request target:

• Direct in-platform and custom OpenAI-compatible models support auto (the default), none, and allowed_tools with mode auto.
• Direct models provided by OpenAI or Azure OpenAI also support required, a named function, and allowed_tools with mode required.
• Preset-agent requests do not accept tool_choice; the agent uses its configured tools.

A named function uses {"type": "function", "name": "your_function"}. To restrict the callable set, use:

{
  "type": "allowed_tools",
  "mode": "auto",
  "tools": [{"type": "function", "name": "your_function"}]
}

Unsupported choices return 400. A named function must appear in the request's tools array; allowed_tools can only select from functions declared there.

Handle unsuccessful responses

Before generation begins, authentication, authorization, validation, and backend connection failures use HTTP error statuses. After a response has begun, the HTTP status can remain 200; inspect the response status and error, or the terminal streaming event, to determine the outcome.

See Model backend errors on the Responses API for the stable client-facing errors and retry guidance.

Compatibility notes

This endpoint is a supported subset of the OpenAI Responses API. In particular:

• only POST /v1/responses is exposed
• background (even when false) and previous_response_id are not currently supported; omit these fields
• platform conversations use conversation_id
• tool_choice support varies by model provider and request target; see tool-choice support
• function and MCP tools are supported; OpenAI-hosted tool types are not
• preset-agent requests accept only user messages, use configured model parameters, and default to store: false
• input, output, tool use, sampling, and multimodal support can vary by model

Use this page as the Model Gateway contract. Fields and operations not documented here are not part of the supported public surface.