OpenAI API Streaming

Streaming sends incremental events over a server-sent event connection. It can improve perceived responsiveness, but receiving some text does not prove the request completed successfully.

#Responses in Python

pythonCopy
import os
from openai import OpenAI

client = OpenAI(timeout=30.0)
stream = client.responses.create(
    model=os.environ["OPENAI_MODEL"],
    input="Explain generators in three short sentences.",
    stream=True,
)
completed = False
try:
    for event in stream:
        if event.type == "response.output_text.delta":
            print(event.delta, end="", flush=True)
        elif event.type == "response.completed":
            completed = True
        elif event.type in ("response.failed", "error"):
            raise RuntimeError(f"Stream failed: {event.type}")
finally:
    stream.close()
if not completed:
    raise RuntimeError("Stream ended without a completed response")

#JavaScript

javascriptCopy
import OpenAI from "openai";
const client = new OpenAI({ timeout: 30000 });
const stream = await client.responses.create({
  model: process.env.OPENAI_MODEL,
  input: "Explain generators in three short sentences.",
  stream: true
});
for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}

The JavaScript fragment demonstrates incremental output; add explicit final/failure handling in an application.

#Existing Chat Completions integrations

Chat Completions can also stream deltas. Inspect optional delta.content, accumulated tool-call arguments and finish_reason; do not execute a tool from an incomplete JSON fragment.

#Application state

Support cancellation, interrupted connections, incremental rendering and a distinct completed/error state. Retrying after partial output needs a deliberate replacement/deduplication policy; appending a new attempt blindly can duplicate text or actions.

Configure a timeout and bounded retries. A retry wrapper around stream creation alone does not handle a failure that occurs later during iteration.

#Reference

• Streaming responses↗