Messages API
Use POST /v1/messages to call a chat model with the Anthropic Messages request and response shapes. The endpoint works with direct Anthropic models and with models served through the OpenAI Responses protocol. The model you choose decides the backend protocol; the endpoint you call decides the shape of the request and response.
This version supports text conversations with images and PDF documents, client tools and tools the gateway runs, as one response or as a stream, and the settings listed below. Other optional settings are not available yet. A request that uses one of them is rejected before it reaches the model.
Before you begin
You need:
- the API URL shown in the platform portal
- an API key with
invokeaccess to a chat model - the model ID from
GET /v1/models
Authenticate with your platform API key as a bearer token: Authorization: Bearer <key>. The x-api-key header is not supported. For an Anthropic model, the gateway sends the connector's Anthropic key; your platform key never reaches Anthropic.
curl -sS "$API_BASE_URL/v1/messages" \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "YOUR_MODEL_ID",
"max_tokens": 1024,
"system": "Answer in one sentence.",
"messages": [{"role": "user", "content": "What is the capital of France?"}]
}'
With the Anthropic Python SDK, set the base URL to the API URL without /v1 and pass the key as an auth token:
import anthropic
client = anthropic.Anthropic(base_url="https://api.<your-domain>", auth_token="cm_api_…")
message = client.messages.create(
model="YOUR_MODEL_ID",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello"}],
)
print(message.content[0].text)
Supported request
| Member | Support |
|---|---|
model | Required. A model or model alias ID. Preset agents are not supported on this endpoint. |
max_tokens | Required integer. null is rejected. 0 is sent to the model as given. |
messages | Required and not empty. Roles user and assistant. A final assistant message (a prefill) returns 400 unless the model is an Anthropic model. |
system | A string, or a list of text blocks joined with a blank line. |
temperature, top_p | Sent to the model when set. |
top_k, stop_sequences, thinking, output_config | See Settings. |
tools, tool_choice | See Tools. |
stream | true or false. See Streaming. |
Message content is a string or a list of blocks. User messages accept text, image, document and tool_result blocks. Assistant messages accept text, thinking, redacted_thinking, tool_use, and cm_gateway mcp_tool_use and mcp_tool_result blocks, so you can send back earlier answers, including their thinking, tool calls and gateway tool activity.
A request that ends with an assistant message asks the model to continue that message. Models served through the Responses protocol cannot continue a message, so the gateway rejects the request instead of returning a new answer. For an Anthropic model the gateway sends the request as given, and the model decides; current Claude models reject a prefilled final assistant message.
Any other request member, including mcp_servers, metadata and service_tier, returns 400. So does any other block type, such as server_tool_use, and any cache_control or non-null citations on a block. The error message names what is not supported.
Images and documents
| Block | Supported sources | Sent to a Responses model as | Sent to an Anthropic model as |
|---|---|---|---|
image | base64 with media_type image/jpeg, image/png, image/gif or image/webp; url | input_image with a data URL or the URL | a native image block |
document | base64 with media_type application/pdf; url | input_file with file_data as a data URL, or file_url | a native document block |
Images and documents are accepted in user messages only. file sources (provider file IDs), text and content document sources, and the document options title, context and citations return 400. The gateway forwards URLs without downloading them; whether the model accepts a given image or PDF is up to the model and its backend.
Settings
The gateway checks these values the same way for every model, before it calls the model. A null effort or format is the same as leaving it out. Otherwise, output_config.effort must be low, medium, high, xhigh or max, and output_config.format must be {"type": "json_schema"} with a schema that is present and not null. Any other value returns 400. The table shows how each accepted setting reaches the model.
| Setting | Anthropic model | Responses model |
|---|---|---|
top_k | Sent as given. Must be an integer. | 400: the Responses protocol has no counterpart. |
stop_sequences | Sent as given. An empty list is the same as none. | A non-empty list returns 400. An empty list is accepted. |
thinking | {"type": "adaptive"}, with an optional display of summarized or omitted, {"type": "disabled"} and {"type": "between_tools"} are sent as given. The model decides which modes it supports. | Any explicit thinking returns 400. Adaptive thinking is not an effort level, so it is never translated. |
output_config.effort | Sent as given. | Sent as reasoning.effort. The model can still reject a value it does not support; only some models accept max. |
output_config.format | Sent as given. | Sent as a strict text.format JSON Schema named output, with the schema unchanged. |
{"type": "enabled", "budget_tokens": …} thinking returns 400: none of the supported Anthropic models accepts a thinking budget, and the gateway never converts a budget into an effort. A budget_tokens member on any other thinking type also returns 400, as do other thinking members besides type and display and other output_config members.
Tools
Declare your own tools in tools. Each tool is {"name", "description", "input_schema", "strict"}, with an optional "type": "custom"; name and an input_schema object are required. A name has 1 to 128 letters, digits, _ or -, and each name is unique. A Responses model accepts names up to 64 characters, so a longer name returns 400 for it. The gateway does not run these tools: the model's calls come back to you as tool_use blocks with stop_reason: "tool_use". Run the tool, then send the assistant content back unchanged followed by a user message with a tool_result block for each call.
| Request | Sent to a Responses model as | Sent to an Anthropic model as |
|---|---|---|
| A tool | A function tool whose parameters is the input_schema unchanged. strict is sent as given, and as false when omitted, since a Responses tool without strict is strict. | A native tool |
tool_choice auto, none | auto, none | Sent as given |
tool_choice any | required | Sent as given |
tool_choice {"type": "tool", "name": …} | A named function choice | Sent as given |
disable_parallel_tool_use | parallel_tool_calls with the opposite value. Omitted, it is not sent. | true is sent as given; false is the default |
Assistant tool_use | A function_call with the same ID, name and input as its arguments | A native tool_use block |
User tool_result | A function_call_output for the same call ID | A native tool_result block |
A forced choice, any or a named tool, is never weakened to auto: a model that cannot honor it returns 400. Most Responses models served by the platform and some Anthropic models do not accept a forced choice. A named tool must be one of the declared tools.
tool_result content is a string or a list of text and image blocks. Within a user message, results and other blocks keep their order. The Responses protocol has no error flag for a function result, so a result with "is_error": true reaches a Responses model as the JSON text {"is_error": true, "content": <the result content>} when its content is a string. Content that is a list of blocks keeps its blocks, so images stay images, after a leading text block {"is_error":true}. An Anthropic model receives is_error natively.
Provider-hosted tool types, such as web search or code execution, return 400, and so do cache_control and other options on a tool. If a declared name is also a gateway tool that you may use, such as a registered MCP tool, the gateway runs it itself; see Gateway tools.
Gateway tools
To let the model use a registered MCP or RAG tool, declare it in tools by its gateway-qualified name, the name the gateway's MCP tools/list returns, with its input schema. The gateway runs each call to a declared gateway tool, after checking that you may use it, and continues the answer with the result. It never adds tools you did not declare. A call to a name you did not declare is not run: the model receives an error result instead.
The response shows each call the gateway handled and its result as MCP blocks of a virtual server named cm_gateway:
[
{"type": "mcp_tool_use", "id": "call_1", "name": "lookup", "server_name": "cm_gateway", "input": {"query": "Paris"}},
{"type": "mcp_tool_result", "tool_use_id": "call_1", "is_error": false, "content": [{"type": "text", "text": "…"}]},
{"type": "text", "text": "Paris is the capital of France."}
]
With Anthropic Python 0.74.0, use client.beta.messages.create() or client.beta.messages.stream() for gateway-tool responses. The beta client understands the MCP blocks. The ordinary unary client preserves their fields but does not type them as MCP blocks; its streaming accumulator loses MCP tool arguments.
Blocks follow the model's turns: a turn's calls come first, then their results, then the next turn. Calls the model made in parallel are therefore followed by all their results, while sequential calls alternate with their results. When a turn also calls your own tools, their tool_use blocks keep their place among the turn's calls, the gateway still runs its own calls, and the response ends with stop_reason: "tool_use". Send your tool_result blocks in the next user message as usual.
By default the gateway makes at most 100 model calls per request, counting the first. A request that finishes, or hands tool calls back to you, on the 100th call succeeds as usual. If the gateway's tool results would need another model call beyond that limit, the request fails with api_error: a unary request returns 500, and a stream ends with an error event instead of message_stop. This limit is separate from max_tokens, which limits the output of each model call.
Send cm_gateway blocks back unchanged in later requests. The gateway turns them into ordinary tool calls and results for the model, keeping each turn and its thinking together; on the next model input, the gateway's results come before your own results and text. Replayed blocks are history only: they are never run again and do not need the tool to still be declared or available. An mcp_tool_use with another server_name, or an mcp_tool_result without an earlier cm_gateway call with its ID, returns 400.
Request-supplied MCP server URLs are not supported, and mcp_servers returns 400.
Response
The response is a Messages message object:
{
"id": "msg_…",
"type": "message",
"role": "assistant",
"model": "YOUR_MODEL_ID",
"content": [{"type": "text", "text": "Paris is the capital of France."}],
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 21,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 9
}
}
model is the model ID you sent. stop_reason is one of:
| Stop reason | Meaning |
|---|---|
end_turn | The model finished its answer. |
max_tokens | The answer reached max_tokens. content holds the partial answer. It can end with a tool_use block that was cut off: its input holds only the members the model completed, and in a stream its input_json_delta events hold the partial JSON. Check stop_reason before you run a call. A call to a gateway tool that was cut off before its arguments were complete is never run, and neither is any other gateway call of that turn: a unary request returns 502 with api_error, and a stream ends with an error event. |
stop_sequence | The model produced one of your stop_sequences. stop_sequence holds the matched string; it is null for every other stop reason. Only Anthropic models accept stop sequences. |
refusal | The model refused. content holds any text the model wrote before it refused, then the refusal explanation as a text block when the model gives one. A model turn that both refuses and calls one of your tools returns an api_error instead, before any gateway tool in that turn runs: 502 for a unary request, an error event in a stream. |
tool_use | The model called your tools. content ends with their tool_use blocks, or with the results of gateway tools the model called in the same turn. |
Any other ending, such as a model failure or a stop reason this API cannot represent, returns an error instead of a message.
usage.input_tokens counts input that was not read from or written to the prompt cache. Cache writes and reads are reported separately; your usage accounting counts all three as input. A cache count the model's backend did not report is null rather than 0. Whether a count is reported depends on the model's backend and the request; many Responses backends do not report cache writes. Usage covers every model turn of the request, including turns that called gateway tools.
Streaming
Set "stream": true to receive the message as server-sent events, in the Messages streaming format:
message_startwith an empty message. Itsusagecounts are placeholders: token counts are0and cache countsnull.- For each content block,
content_block_start, itscontent_block_deltaevents andcontent_block_stop.indexcounts the blocks of the response. Text arrives astext_delta, thinking asthinking_deltafollowed by onesignature_delta. Aredacted_thinkingblock arrives whole in itscontent_block_start. Atool_useblock starts withinput: {}, and its input arrives asinput_json_deltaevents whosepartial_jsonstrings join into the input's JSON text. Anmcp_tool_useblock streams the same way, and anmcp_tool_resultblock arrives whole in itscontent_block_start. Gateway tool blocks arrive as each turn and its tool calls finish, before the answer is complete. To accumulate MCP blocks with the Anthropic Python SDK, see Gateway tools. message_deltawithstop_reason,stop_sequenceand the finalusage, totaled over the whole request and equal to the unary response's, thenmessage_stop.
With the Anthropic Python SDK, client.messages.stream() sends the request with stream set and accumulates the events:
with client.messages.stream(
model="YOUR_MODEL_ID",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello"}],
) as stream:
for text in stream.text_stream:
print(text, end="")
message = stream.get_final_message()
The gateway sends ping events while the model is silent. The blocks, stop reason and usage you accumulate from the stream equal the unary response for the same answer, with one exception. When a Responses model sends a reasoning item only in its finished response, after answer text has already streamed, the stream delivers that thinking block after the text, while the unary response keeps the model's order. The block's thinking and signature are the same in both. The gateway has not confirmed that a model accepts the reasoning when it is sent back in this changed order.
With an Anthropic model, the gateway learns that a block is complete only when the model's message ends. Block starts and deltas arrive as the model produces them, but every block's signature_delta and content_block_stop arrive after the last block has started, just before message_delta. Deltas for an earlier block can therefore follow the start of a later one. Accumulate deltas by index, as the Anthropic SDKs do. When an Anthropic model refuses, the stream and the unary response both hold the text written before the refusal. Thinking already streamed also stays in the stream, without a signature, while the unary response leaves it out. A tool_use block streamed before the refusal cannot be withdrawn, so the stream ends with an error event, while the unary response leaves the call out and returns the refusal. The TypeScript SDK's per-event callbacks, such as signature and contentBlock, can name the wrong block in this order, but finalMessage() and the accumulated message are correct.
Errors found before the stream starts, such as an invalid request or a denied model, return an ordinary JSON error with its HTTP status. An error after the stream has started ends it with one error event carrying the same error object, and no message_stop. The stream starts before the gateway calls the model, so when the model service rejects the request, for example with 429, the stream ends with an error event: the HTTP status is already 200, and no Retry-After header is sent. If the client disconnects, the gateway stops the model request.
Conversation history
The endpoint is stateless. It stores no conversation and creates no conversation items. To continue a conversation, send the earlier messages again, including assistant content exactly as you received it.
Assistant thinking and redacted_thinking blocks must be sent back unchanged. Gateway-added context, such as the description of the tools the model may call, is part of the model input but is not returned. If that context changes between requests, the model may reject earlier thinking.
Thinking from Responses models
Messages has no field for the reasoning items that OpenAI Responses models return. The gateway returns each one as a thinking block: its thinking is the reasoning summary followed by the reasoning text, separated by blank lines, and its signature starts with cmgw. and carries the reasoning item. Send the block back unchanged; the gateway restores the original reasoning item for the model. A cmgw. signature is not an Anthropic signature: sending it to an Anthropic model returns 400. A malformed cmgw. signature also returns 400.
Errors
Errors use the Messages error shape:
{"type": "error", "error": {"type": "invalid_request_error", "message": "max_tokens is required"}}
| Status | error.type |
|---|---|
400 | invalid_request_error |
401 | authentication_error |
403 | permission_error |
404 | not_found_error |
413 | request_too_large |
429 | rate_limit_error |
529 | overloaded_error |
| Other | api_error |
A missing or invalid credential is rejected before the request reaches the gateway and does not use the Messages error shape: a missing or unrecognized credential returns 403, and a recognized API key that is invalid returns 401.
When the model service rejects a request that does not stream, the gateway returns its status with the public message described in Model backend errors, and forwards a Retry-After header. Provider messages are not returned.
When the model service fails after it started answering, the gateway returns the same public message. An invalid request, including one too long for the model, returns 400; an authentication failure 401; a permission failure 403; and a rate limit 429, without a Retry-After header. Any other failure returns 502 with api_error.