Streaming

Read the response as it arrives instead of waiting for the full answer. Chat Completions streams use server-sent events (SSE); the SDK parses those events into chunks you can iterate over.

Stream a response

These examples use the OpenAI SDK with the Chat Completions endpoint. Set STRAITLY_API_KEY in your terminal or server environment first. Install openai for Python or Node.js; the SDKs guide covers installation and running TypeScript.

Set stream to true. Each text delta contains only the next piece of the answer, so append it rather than replacing the previous text.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.straitly.ai/v1",
    api_key=os.environ["STRAITLY_API_KEY"],
)

with client.chat.completions.create(
    model="claude-sonnet-5",
    messages=[{"role": "user", "content": "Count to ten."}],
    stream=True,
    stream_options={"include_usage": True},
) as stream:
    for chunk in stream:
        if chunk.choices:
            text = chunk.choices[0].delta.content
            if text:
                print(text, end="", flush=True)
        if chunk.usage is not None:
            print("\nUsage:", chunk.usage)
print()

Read the chunks

01 / Receive

Text deltasAppend choices[0].delta.content when present. Some chunks contain role metadata or tool-call deltas instead of text.

02 / Finish

Finish reasonInspect finish_reason when it is present. A length limit or tool call is not the same as a normal completed text answer.

03 / Account

UsageWith include_usage enabled, a usage chunk can have an empty choices array. Keep reading after the last text delta.

Do not assume every chunk has text or that choices[0] exists. A connection closing early is not proof that generation finished successfully.

Usage and cost

Ask for stream_options.include_usage to receive token usage in the stream. Earlier chunks may have no usage data. If the connection is interrupted, you may not receive the final usage chunk.

Use Activity to inspect recorded token buckets and the final charge. Do not estimate your invoice from the number of streamed chunks: one chunk is not one token.

Tools and other formats

• Tool calls can arrive across several deltas. Accumulate the tool name and arguments by tool-call index; do not parse a partial JSON argument string or execute an incomplete call.
• Structured output may be incomplete until generation ends. Validate the assembled result before using it.
• The Anthropic Messages API has its own event types. Use the Anthropic SDK's stream handling instead of the choices/delta loop shown above.
• Tools, images, caching, and structured outputs depend on the selected model and request format. Streaming does not add capabilities to a model.

Disconnects and retries

• Handle exceptions around both stream creation and iteration. Authentication or balance errors can occur before streaming starts; network failures can occur after partial output.
• If output stops halfway through, show it as incomplete. A new request is a new generation, not a continuation of the previous stream.
• Avoid automatically repeating tool actions when retrying a generation. Your application must guard side effects against duplicate execution.
• When the user cancels, close or abort the upstream stream too. Check Activity for recorded usage rather than assuming that cancellation makes a request free.

Streaming through your server

Keep your Straitly key on the server and relay output to your frontend. Forward chunks as they arrive; collecting the entire response before sending it defeats streaming.

Check your reverse proxy and hosting platform for response buffering and request timeouts. If text arrives all at once, test the API directly before debugging the frontend.

curl -N https://api.straitly.ai/v1/chat/completions \
  -H "Authorization: Bearer $STRAITLY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [{"role": "user", "content": "Count to ten."}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

curl -N disables curl's output buffering. Raw Chat Completions SSE uses data: events and a [DONE] marker. Network reads are not necessarily complete SSE events; use an SSE parser or an SDK.