Configure VS Code Agent Mode

VS Code agent mode can use models hosted on ConfidentialMind through the Custom Endpoint language model provider. This guide shows the recommended configuration and explains what VS Code expects from the endpoint, so you can tell expected behavior on long tasks from a configuration problem.

Before you begin

You need:

• A recent VS Code release with the Custom Endpoint language model provider. The older OpenAI Compatible provider is deprecated.
• Your Model Gateway domain — shown as API URL at the bottom of the portal sidebar, next to a button that copies it.
• A personal API key with invoke granted on the model you want to use.
• A chat model that supports tool calling and is suitable for coding tasks.
• The model's served limits: its context window and the maximum output tokens configured on your deployment. For a model deployed on the platform, its page under Models in the portal shows the served context window as Context length.

1. Find the model ID

List the chat models your key is allowed to invoke:

curl -sS "https://api.YOUR_DOMAIN/v1/models" \
  -H "Authorization: Bearer $CM_MODEL_API_KEY"

Choose the exact id of a coding-capable model from the response. The display name is not interchangeable with the ID.

2. Add the Custom Endpoint provider

1. Run Chat: Manage Language Models from the Command Palette.
2. Choose Add Models and select Custom Endpoint.
3. Enter your API key when prompted. VS Code stores it in its secret storage and refers to it from the configuration file as ${input:…}, so the key does not appear in plain text.

VS Code saves the provider in its language models file. Open it with Chat: Open Language Models (JSON) and set the model entry as follows. Replace YOUR_DOMAIN and MODEL_ID, and keep the apiKey value that VS Code wrote.

[
  {
    "name": "CM Gateway",
    "vendor": "customendpoint",
    "apiKey": "${input:chat.lm.secret.…}",
    "models": [
      {
        "id": "MODEL_ID",
        "name": "Qwen3.6 27B via CM Gateway",
        "url": "https://api.YOUR_DOMAIN/v1/chat/completions",
        "apiType": "chat-completions",
        "toolCalling": true,
        "vision": false,
        "contextWindow": 262144,
        "maxOutputTokens": 32768
      }
    ]
  }
]

The limits in this example are the tested values for Qwen3.6 27B. For a different model, use the Context length shown on its page in the portal, and the output limit configured on your deployment.

Field	Recommended value	Why
url	https://api.YOUR_DOMAIN/v1/chat/completions	The gateway's OpenAI-compatible Chat Completions API.
apiType	chat-completions	See Chat Completions or Responses.
toolCalling	true	Agent mode only offers models that support tool calling.
contextWindow	The served context window, in tokens	VS Code plans every request against this value. Do not set it higher than the deployment allows.
maxOutputTokens	The served output limit, in tokens	VS Code reserves this many tokens for the reply.
vision	true only for models that accept images	VS Code sends images only to models marked as supporting them.

3. Verify the connection

Open the Chat view, switch to Agent mode, and select the model under CM Gateway in the model picker. Ask a short question such as "Reply with: ConfidentialMind gateway reached". Then give the agent a small task that reads a file in your workspace, to confirm that tool calls work.

How VS Code manages conversation context

VS Code runs the agent loop and owns the conversation. On every step it builds the full prompt — instructions, conversation history, earlier tool calls and their results — and sends it to the gateway. The gateway does not yet pick the conversation up from an earlier response or shorten it for you, so VS Code manages the context on its side. The platform can store conversations for the Responses API, but VS Code does not use that.

To keep the prompt inside the model's limits, VS Code does the following on its side:

• It plans each request against contextWindow minus maxOutputTokens, and keeps a safety margin for tool definitions.
• When the conversation approaches that limit, it replaces older steps with a summary written by the model. Exact file contents and details from before the summary can be lost, so on long tasks the agent may read a file again or ask about something it saw earlier. This is expected behavior.
• It shortens very long tool results, such as large file reads or command output.

Because these limits come from your configuration, they decide how VS Code behaves:

• If the configured limits are higher than the served model allows, VS Code does not summarize in time and the model rejects the request as too long.
• If they match the served limits, long tasks are summarized before they overflow.
• Models with smaller context windows reach the limit sooner and are summarized more often.

Chat Completions or Responses

Use chat-completions for VS Code agent mode.

The gateway also serves the Responses API, but some VS Code features for Responses endpoints rely on server-side conversation state that the gateway does not yet support:

• previous_response_id is not yet supported. Until it is, every request must carry the full conversation.
• Server-side compaction is not yet supported. If VS Code asks the server to compact the conversation, it also turns off its own summarization, and long tasks then run out of context.

If you must use responses, add these settings so that VS Code keeps managing the context itself:

• In the model entry, set "statefulResponses": false and use https://api.YOUR_DOMAIN/v1/responses as the url.
• In your VS Code user settings, set "github.copilot.chat.responsesApiContextManagement.enabled": false.

Troubleshooting

Symptom	Check
Request returns 401	The API key is unexpired, and neither revoked nor disabled. Re-enter it with Chat: Manage Language Models if it changed.
Request returns 403	The key has invoke granted on the configured model.
Model not found	id exactly matches an id returned by GET /v1/models, and the model is ready.
The model does not appear in agent mode	toolCalling is true for the model.
The model writes tool calls as text	Select a coding or instruction model with tool-calling support.
The request fails because the prompt is too long	contextWindow and maxOutputTokens match the served limits, not larger values.
The agent forgets files or details from earlier in a long task	VS Code summarized older steps; see How VS Code manages conversation context. Split long work into smaller tasks, start a new chat per task, or attach the files the agent needs again.
Long tasks run out of context with apiType set to responses	Switch to chat-completions, or apply the settings in Chat Completions or Responses.
"Sorry, no response was returned."	VS Code received no usable reply. Either the response stream ended before the model finished, or VS Code discarded a reply that kept repeating itself. Select Try again. If it repeats, note the time of the failure and contact your administrator so the gateway logs can be checked.

A disabled key can be enabled again from the API Keys page.

To give agents access to gateway-hosted RAG tools through MCP, see Configure Coding Agents.