Streaming, errors, and retries

Parse Singular's Chat Completions SSE stream correctly and retry only failures that can reasonably recover.

ReferenceAvailability: Stable

On this page

Streaming request
JavaScript stream
Python stream
Error envelope
Retry pattern

Streaming request

Set stream: true on Chat Completions:

bash

curl -N "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $SINGULAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "stream": true,
    "messages": [{"role": "user", "content": "Count to three."}]
  }'

The response is a data-only Server-Sent Events stream. Each payload is a Chat Completions chunk; partial text is under choices[].delta.content. A normal stream ends with:

text

data: [DONE]

stream_options is not part of Singular's stable forwarded parameter subset. Do not require OpenAI's optional final usage-only chunk.

JavaScript stream

javascript

const stream = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Count to three." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Python stream

python

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Count to three."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="")

Error envelope

Singular-generated API errors commonly use this envelope:

json

{
  "status": "failure",
  "code": "too_many_requests",
  "message": "Too many requests",
  "retryAfterSeconds": 30
}

Sanitized upstream failures can instead preserve an OpenAI-shaped {"error":{"message":"...","type":"...","code":"..."}} object, while some middleware validation failures use the shorter {"status":"failure","message":"..."} form. Clients must branch on HTTP status first and tolerate either family, a missing code, or a short 400 body.

HTTP status	Meaning	Retry?
`400` / `422`	Invalid request or unsupported shape	No; fix the request.
`401`	Missing or invalid key	No; replace credentials.
`402`	Payment or balance action required	No blind retry; satisfy the advertised challenge or fund the key.
`403`	Revoked key or authenticated caller not permitted	No; replace the key or change policy/balance.
`404`	Unknown resource or unavailable surface	No.
`408`	Request timeout	Usually, with a bound.
`429`	Rate limited	Yes; honor `Retry-After`.
`5xx`	Singular service or upstream failure	Usually, with exponential backoff and jitter.

A 401 response includes WWW-Authenticate: Bearer. A 429 may include Retry-After; the gateway supplies a fallback value when a more specific limit does not.

Retry pattern

python

import random
import time
from openai import APIStatusError, RateLimitError

def call_with_retry(call, attempts=5):
    for attempt in range(attempts):
        try:
            return call()
        except (RateLimitError, APIStatusError) as error:
            status = getattr(error, "status_code", None)
            if status not in (408, 429, 500, 502, 503, 504) or attempt == attempts - 1:
                raise
            time.sleep((2 ** attempt) + random.random())

Use an application-level idempotency strategy before retrying workflows that trigger side effects outside the model call.