# Streaming

> Stream ScaleMax OpenAI-compatible and Anthropic Messages responses, parse server-sent events, and handle disconnects safely.

POST `/v1/chat/completions` (API credit) or `/token/v1/chat/completions` (API token) with `stream: true`

ScaleMax streams model responses using **server-sent events** (SSE) — the same wire format as OpenAI. Pass `"stream": true` in the request body and consume the response as a stream.

For OpenAI-compatible clients, set the base URL to `https://api.scalemax.pro/v1` for API credit or `https://api.scalemax.pro/token/v1` for API tokens before using this endpoint.

## Why stream

- **Lower time-to-first-token.** Users see text as soon as the model produces it.
- **Better UX for long responses.** Avoid a 30-second blank screen.
- **Cancelable.** Disconnect mid-response to stop usage charges for tokens you won't use.

## Request

Same as a non-streaming chat completion, with one extra flag:

**macOS / Linux**

stream.sh:

```bash
curl -N "$SCALEMAX_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $SCALEMAX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "YOUR_AUTHENTICATED_MODEL_ID",
    "stream": true,
    "messages": [{"role": "user", "content": "Write a haiku about streaming APIs."}]
  }'
```

**Windows PowerShell**

stream.ps1:

```powershell
@'
{
  "model": "YOUR_AUTHENTICATED_MODEL_ID",
  "stream": true,
  "messages": [{"role": "user", "content": "Write a haiku about streaming APIs."}]
}
'@ | Set-Content -Path body.json

curl.exe -N "$env:SCALEMAX_BASE_URL/chat/completions" `
  -H "Authorization: Bearer $env:SCALEMAX_API_KEY" `
  -H "Content-Type: application/json" `
  --data-binary "@body.json"
```

PowerShell changes quotes inside a JSON argument, so the request body goes into body.json first and curl.exe sends that file. Type curl.exe, not curl: in Windows PowerShell 5.1, curl is a different command.

The `-N` flag disables curl's output buffering so you see chunks arrive in real time.

## Response

A series of SSE events. Each event is a JSON object on a single line prefixed with `data:`. The stream ends with `data: [DONE]`.

events:

```text
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1719228000,"model":"fable-5","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1719228000,"model":"fable-5","choices":[{"index":0,"delta":{"content":"Tokens"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1719228000,"model":"fable-5","choices":[{"index":0,"delta":{"content":" fall"},"finish_reason":null}]}

data: [DONE]
```

## With the OpenAI SDK

The SDK handles SSE for you. Just set `stream=True`:

stream.py:

```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["SCALEMAX_API_KEY"],
    base_url=os.environ["SCALEMAX_BASE_URL"],
)

stream = client.chat.completions.create(
    model="YOUR_AUTHENTICATED_MODEL_ID",
    stream=True,
    messages=[{"role": "user", "content": "Write a haiku about streaming APIs."}],
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)
print()
```

## Cancellation

Disconnect the request to cancel. The provider stops generating, partial tokens are billed, no usage record is written for the cancelled call. (Implemented via HTTP connection close — the gateway sees the TCP RST and forwards it upstream.)

## Notes

- Streams are **not buffered** by the gateway. Bytes from the provider go straight to your client.
- The gateway sets `Cache-Control: no-store` on streaming responses.
- Request timeouts on streaming are 5 minutes. Longer streams may need a server-side reconnect.

Working streaming clients: [Python](https://docs.scalemax.pro/docs/examples/python), [JavaScript](https://docs.scalemax.pro/docs/examples/javascript), and [cURL](https://docs.scalemax.pro/docs/examples/curl). Failure codes returned mid-stream are listed in [Error Codes](https://docs.scalemax.pro/docs/error-codes).