Streaming

Set stream: true on a chat completion to receive the reply as server-sent events. The stream ends with data: [DONE], as in the OpenAI API, so the OpenAI SDKs’ streaming helpers work unchanged. The Anthropic Messages endpoint streams Anthropic-style events when you set stream: true there.

Python (openai)

from openai import OpenAI

client = OpenAI(base_url="https://api.lab.litepod.io/v1", api_key="sk_modeldock_...")

stream = client.chat.completions.create(
    model="modeldock/<your-model-id>",
    messages=[{"role": "user", "content": "Write a haiku about GPUs."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

curl

curl -N https://api.lab.litepod.io/v1/chat/completions \
  -H "Authorization: Bearer $MODELDOCK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "modeldock/<your-model-id>", "stream": true, "messages": [{"role": "user", "content": "Hi"}]}'

Rate limits and Pre-flight caps still apply to streamed requests. See Errors and limits.

More from litepod: the free LLM API with free open-source models. ModelDock is a litepod product.