Server-Sent Events

Streaming

Stream responses incrementally using server-sent events (SSE). By setting stream: true, the application processes delta events as they arrive, significantly reducing time to first token. Streaming is available only for models and endpoints that support it.

Request

SSE connection

Delta events

Completed response

Examples

from openai import OpenAI

openai = OpenAI(
    api_key="$DEVUP_API_KEY",
    base_url="https://api.devupai.com/v1",
)

stream = openai.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Pro",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content is not None:
        print(content, end="", flush=True)
print()

SSE format

Each streamed chunk is a data: event containing a JSON payload. The text delta normally arrives through choices[0].delta.content. Every chunk of a stream carries the same id, in the form chatcmpl- followed by a UUID. When the model completes its reply, the last chunk has an empty choices array and carries usage; no other chunk carries usage. The stream ends when the server closes the SSE connection. OpenAI-compatible clients may also receive a data: [DONE] terminal event.

Tool calls arrive in choices[0].delta.tool_calls, one call per index, with the arguments in parts; see Streaming tool calls.

Illustrative stream output. The data: [DONE] terminal event may not be present in every stream.

json

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":" World"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":11,"completion_tokens":2,"total_tokens":13}}

data: [DONE]

Errors and cancellation

Stopping a stream

The client may abort the HTTP request at any time using an AbortController (in JS) or the SDK's supported cancellation mechanism.

When you stop a stream, the input and the output sent up to that point are billed. Output includes text, reasoning and tool calls. A request stopped before the response starts, that is before the HTTP status and headers arrive, is not billed.

Stream errors

Before the first chunk. When the model returns an error before any output, no stream is opened: the response is an HTTP error with the same JSON body as a request without streaming, for example HTTP 400 context_length_exceeded.

During the stream. An error is sent as one event, data: {"error": {"message": …, "type": "devup_error", "code": …}}, followed by data: [DONE]. Nothing follows it. Output received before it is kept by your client.

Billing. A request that ends with an error is not billed, and neither is a stream that ends without any output. Applications should also handle network interruptions, keeping in mind that partial output may have already been received.

What's Next