# AI chat streaming API

> Reference: DCS chat endpoints, stream payloads, toolset selection, model identifiers, extraction statuses, structured completion, and stop semantics.

- Repository: macro-inc/macro
- GitHub: https://github.com/macro-inc/macro
- Human docs: https://grok-wiki.com/public/docs/macro-inc-macro-bb988e1a448e
- Complete Markdown: https://grok-wiki.com/public/docs/macro-inc-macro-bb988e1a448e/llms-full.txt

## Source Files

- `js/app/packages/service-clients/service-cognition/openapi.json`
- `js/app/packages/service-clients/service-cognition/client.ts`
- `rust/cloud-storage/document_cognition_service/src/model/stream.rs`
- `rust/cloud-storage/document_cognition_service/src/api/stream/chat_message/mod.rs`
- `rust/cloud-storage/document_cognition_service/src/api/stream/stop.rs`
- `rust/cloud-storage/agent/src/model.rs`
- `js/app/packages/core/component/AI/types/index.ts`

---

---
title: "AI chat streaming API"
description: "Reference: DCS chat endpoints, stream payloads, toolset selection, model identifiers, extraction statuses, structured completion, and stop semantics."
---

DCS exposes chat generation as an HTTP initiation API plus durable stream delivery through `connection_gateway`; `POST /stream/chat/message` returns a `stream_id` and `chat_id`, then clients subscribe to the chat stream to receive `ChatStream` payloads until `stream_end`.

## Runtime shape

```text
UI / service-cognition client
  POST /stream/chat/message
        │ returns { stream_id, message_id, chat_id }
        ▼
DCS stream repo: id = { entity_type: "chat", entity_id: chat_id, stream_id }
        │ publishes payloads
        ▼
connection_gateway websocket
        │ message.type = "stream"
        ▼
@service-connection/stream.subscribe("chat", chat_id, stream_id)
```

<Note>
The chat endpoint is not an SSE response. The HTTP response only acknowledges stream creation. Chunks arrive later as `connection_gateway` websocket stream events.
</Note>

## Endpoints

:::endpoint POST /stream/chat/message Initiate a streamed chat response

Creates or reuses a chat, stores the incoming user message, starts the agent loop, and publishes stream payloads under the returned `stream_id`.

### Request body

| Field | Type | Required | Behavior |
| --- | --- | --- | --- |
| `content` | `string` | Yes | User message text. |
| `model` | `AgentModel` | Yes | Required by schema. In the current chat message handler, the effective response model is resolved from `ChatModelAccess` rather than read directly from this field. |
| `chat_id` | `string \| null` | No | Existing chat ID. If omitted, empty, or not found, DCS creates a new persistent chat named with the default chat name. |
| `attachments` | `Entity[] \| null` | No | Entities attached to the message. Resolved attachment parts from the current and prior message chain are included in the prompt. |
| `additional_instructions` | `string \| null` | No | Appended after the selected tool prompt. Frontend chat input also appends a model instruction string. |
| `toolset` | `ToolSet` | No | Defaults to `{ "type": "all" }`. See [Toolset selection](#toolset-selection). |

<RequestExample>

```http
POST /stream/chat/message
Content-Type: application/json

{
  "content": "Summarize the attached document.",
  "model": "smart",
  "chat_id": "chat_123",
  "attachments": [
    {
      "entity_type": "document",
      "entity_id": "doc_456"
    }
  ],
  "toolset": { "type": "all" },
  "additional_instructions": "Prefer concise bullets."
}
```

</RequestExample>

### Response body

| Field | Type | Behavior |
| --- | --- | --- |
| `stream_id` | `string` | Stream identifier for `connection_gateway` subscription. Generated server-side. |
| `message_id` | `string` | Assistant message ID. The implementation sets this to the same value as `stream_id`. |
| `chat_id` | `string` | Actual chat ID. May differ from the request if DCS created a new chat. |

<ResponseExample>

```json
{
  "stream_id": "0de0fb91-3f54-4680-97a0-f3e97b9a2a69",
  "message_id": "0de0fb91-3f54-4680-97a0-f3e97b9a2a69",
  "chat_id": "chat_123"
}
```

</ResponseExample>

### Client helper

`cognitionApiServiceClient.sendStreamChatMessage(args)` posts to `/stream/chat/message`. The chat input flow then calls:

```ts
subscribe('chat', chat_id, stream_id)
```

and treats `stream_end` as the terminal marker.

:::endpoint POST /stream/chat/message/stop Stop an in-flight chat stream

Publishes a cancellation signal for an in-flight stream. The caller must have more than `View` or `Comment` access to the chat.

### Request body

| Field | Type | Required | Behavior |
| --- | --- | --- | --- |
| `chat_id` | `string` | Yes | Chat used for permission checks. |
| `stream_id` | `string` | Yes | Stream/message ID returned by `/stream/chat/message`. |

<ResponseExample>

```json
{
  "stopped": true
}
```

</ResponseExample>

`stopped: false` means no matching live subscriber received the cancel signal, usually because the stream already finished or the serving DCS instance is gone.

:::endpoint POST /structured-completion Run a two-phase structured completion

Runs an agent loop to gather context, then asks the model to return JSON matching `output_schema`.

### Request body

| Field | Type | Required | Behavior |
| --- | --- | --- | --- |
| `prompt` | `string` | Yes | User prompt for the information-gathering phase. |
| `model` | `AgentModel` | Yes | Model passed to the agent loop and structured output call. |
| `output_schema` | `DynamicSchema` | Yes | `{ name, schema, description? }`; `schema` is the JSON schema to validate against. |
| `additional_instructions` | `string \| null` | No | Appended to the selected tool prompt. |
| `toolset` | `ToolSet` | No | Defaults to all-tool prompt behavior. |

<ResponseExample>

```json
{
  "result": {
    "summary": "The document describes renewal terms.",
    "risk_level": "medium"
  }
}
```

</ResponseExample>

## Stream event envelope

`connection_gateway` websocket messages use `type: "stream"` and carry a serialized stream item:

```json
{
  "id": {
    "entity_type": "chat",
    "entity_id": "chat_123",
    "stream_id": "0de0fb91-3f54-4680-97a0-f3e97b9a2a69"
  },
  "payload": {
    "type": "chat_message_response",
    "stream_id": "0de0fb91-3f54-4680-97a0-f3e97b9a2a69",
    "chat_id": "chat_123",
    "message_id": "0de0fb91-3f54-4680-97a0-f3e97b9a2a69",
    "content": {
      "type": "text",
      "text": "Here is the summary..."
    }
  }
}
```

The frontend stream store appends payloads to the stream unless `payload.type === "stream_end"`, in which case it marks the stream done.

## ChatStream payloads

| `type` | Fields | Notes |
| --- | --- | --- |
| `chat_user_message` | `stream_id`, `chat_id`, `message_id`, `content`, `attachments` | First item emitted by the HTTP chat message handler so other clients can display the user message. |
| `chat_message_response` | `stream_id`, `message_id`, `chat_id`, `content` | Repeated assistant chunks. `content` is an `AssistantMessagePart`. |
| `stream_end` | `stream_id` | Terminal event. Stop and normal completion both end with this marker. |
| `error` | `stream_error` payload | Emitted for stream creation errors, AI stream errors, and idle timeout. |
| `chat_message_ack` | `message_id`, `chat_id` | Defined in the shared stream schema. |
| `chat_message_finished` | `message_id`, `chat_id`, `user_message_id` | Defined in the shared stream schema. |
| `chat_renamed` | `stream_id`, `chat_id`, `name` | Defined in the shared stream schema. |
| `model_selection_changed` | `chat_id`, `available_models`, `new_model?` | Defined in the shared stream schema. |
| `token_count_changed` | `chat_id`, `token_count` | Defined in the shared stream schema. |
| `chat_message_status_update` | `chat_id`, `message_id`, `message` | Defined in the shared stream schema. |
| `extraction_status_ack` | `attachment_id`, `status` | Initial extraction status response. |
| `extraction_status_update` | `attachment_id`, `status` | Later extraction status update. |
| `completion_response` | `completion_id`, `content`, `done` | PDF completion payload. |
| `completion_stream_chunk` | `completion_id`, `content`, `done` | PDF completion chunk payload. |

### Assistant message parts

`chat_message_response.content` uses camelCase part tags:

| `content.type` | Fields |
| --- | --- |
| `text` | `text` |
| `thinking` | `thinking` |
| `toolCall` | `name`, `json`, `id` |
| `mcpToolCall` | `name`, `service`, `display_name`, `json`, `id` |
| `toolCallResponseJson` | `name`, `json`, `id` |
| `toolCallErr` | `name`, `description`, `id` |

Frontend message assembly ignores non-`chat_message_response` payloads and merges adjacent `text` chunks and adjacent `thinking` chunks into single parts.

## Toolset selection

`ToolSet` is a tagged object:

```json
{ "type": "all" }
```

```json
{ "type": "none" }
```

| Toolset | Prompt selected |
| --- | --- |
| `all` | DCS `all_tools_prompt` |
| `none` | `ai_tools::prompts::BASE_PROMPT` |

<Warning>
In the current chat and structured-completion implementations, `toolset` selects the system prompt. The agent session is still constructed with the combined static and MCP toolset from enabled MCP records.
</Warning>

## Model identifiers

Use `AgentModel` values in API payloads and frontend state, not provider API IDs.

| API value | UI label | Provider route in agent model |
| --- | --- | --- |
| `smart` | Smart | Anthropic Opus 4.7 |
| `fast` | Fast | Anthropic Haiku 4.5 |
| `opus4_7` | Opus 4.7 | Anthropic Opus 4.7 |
| `sonnet4_6` | Sonnet 4.6 | Anthropic Sonnet 4.6 |
| `haiku4_5` | Haiku 4.5 | Anthropic Haiku 4.5 |
| `retired` | Retired | Falls back to the Smart/Opus route |

Additional model behavior:

| Model group | Context window | Thinking parameters |
| --- | ---:| --- |
| `smart`, `opus4_7`, `sonnet4_6`, `retired` | `1_000_000` tokens | Opus uses adaptive thinking; Sonnet uses enabled thinking with `budget_tokens: 10000`. |
| `fast`, `haiku4_5` | `200_000` tokens | Enabled thinking with `budget_tokens: 10000`. |

Unrecognized model strings deserialize to `retired`, which routes to the default Opus-backed path.

## Extraction statuses

Extraction statuses are tagged objects:

```json
{ "type": "incomplete" }
```

| Status | Meaning |
| --- | --- |
| `incomplete` | No extracted document text row exists yet. |
| `empty` | Extraction completed, but extracted text is empty or has no content length. |
| `insufficient` | Extracted text has fewer than `1000` non-whitespace characters. |
| `complete` | Extracted text has at least `1000` non-whitespace characters. |

`extraction_status_ack` tells the client the current status. If it is `incomplete`, clients should wait for later `extraction_status_update` events for the same `attachment_id`.

## Stop and persistence semantics

Stopping a stream publishes to Redis on a per-`stream_id` cancellation channel. The DCS instance running the stream subscribes when generation starts; any instance can publish the stop signal.

When cancellation is observed:

1. The streaming loop breaks.
2. DCS emits `stream_end`.
3. DCS persists the assistant parts already yielded.
4. If a yielded tool call has no matching response, DCS inserts a persisted `toolCallErr` with `description: "cancelled"` so later conversation turns remain well-formed.
5. DCS skips the notification summarization path for cancelled streams.

The chat stream also has a `3 minute` idle timeout between AI stream items. Timeout emits an `error` payload with `internal_error`, then the stream ends.

## Error responses

| Surface | Statuses in API schema | Body |
| --- | --- | --- |
| `POST /stream/chat/message` | `400`, `401`, `402`, `403` | `400` returns `{ "error": string, "stream_id"?: string }`. |
| `POST /stream/chat/message/stop` | `200`, `403` | `403` returns `{ "error": string }`. |
| `POST /structured-completion` | `400`, `401`, `402`, `500` | Error responses return `{ "error": string }`. |

Common failure points include invalid user IDs, missing chat permissions, message storage failure, request building failure, agent stream creation failure, AI stream errors, and structured-output validation or generation failure.

## Related pages

- Chat state and stream rendering
- Connection gateway stream subscription
- MCP tool registration and tool call rendering