Streaming Behavior

Streaming Behavior

When a client asks ProxAI for a streamed response, ProxAI preserves the upstream Server-Sent Events (SSE) byte stream while observing protocol events. From a user perspective, three things matter: the stream eventually ends, errors arrive in a readable shape, and stalled tool calls do not hang forever.

You do not need to learn ProxAI internals

This page describes observable streaming behavior. For internal streaming architecture, see Streaming Internals.

What a healthy stream looks like

Each provider protocol has its own terminal event. ProxAI considers a stream complete only when the expected terminal event arrives. This implements Behavior Contract C19.

OpenAI Responses

1. 1
   response.created

   Stream starts.

2. 2
   response.output_item.added

   A new output item begins (message, reasoning, function_call, etc.).

3. 3
   response.output_text.delta / response.function_call_arguments.delta

   Incremental text or tool-call argument bytes.

4. 4
   response.output_item.done

   Output item is complete.

5. 5
   response.completed

   Terminal event. The stream is complete.

OpenAI Chat Completions

1. 1
   chat.completion.chunk

   Delta chunks for content, tool calls, and finish_reason.

2. 2
   [DONE]

   Terminal sentinel. Without it, EOF is treated as an incomplete stream.

Anthropic Messages

1. 1
   message_start

   Stream starts.

2. 2
   content_block_start / content_block_delta / content_block_stop

   A content block (text, thinking, tool_use, etc.) is produced incrementally.

3. 3
   message_delta

   Carries stop_reason, stop_sequence, usage, and other message-level deltas.

4. 4
   message_stop

   Terminal event. The stream is complete.

Stalled tool calls

If a provider starts streaming function/tool arguments but never finishes them, ProxAI can close the stream with a protocol-shaped error instead of hanging the client forever. This is controlled by:

[tool_calls]
timeout_secs = 120

The timeout is semantic, not a total request timeout. It only applies after tool-call argument streaming has started. This implements Behavior Contract C20.

Common symptoms

Stream hangs during a tool call

Likely causes

• Upstream started tool arguments but did not send the matching done event.
• Provider stalled while the raw HTTP stream stayed open.

Next checks

• Lower or tune `[tool_calls].timeout_secs`.
• Enable `upstream_response` capture for one request.
• Check the recent SSE tail in diagnostics.

Stream closes with no final answer

Likely causes

• Missing terminal event such as `[DONE]`, `response.completed`, or `message_stop`.
• Upstream connection closed before the protocol state was complete.

Next checks

• Check stream outcome logs.
• Compare the provider event timeline with the expected terminal event.
• Capture the upstream response bytes.

Unicode text looks corrupted

Likely causes

• You are interpreting partial byte chunks as complete UTF-8 text.
• A tool is slicing raw bytes as a string.

Next checks

• Decode only complete SSE event data.
• Keep scanning byte-oriented.
• Avoid direct string slicing on arbitrary chunks.

Related docs

ConfigurationTune `[tool_calls].timeout_secs` and other runtime settings.ObservabilityCapture and logs for debugging streaming failures safely.TroubleshootingBroader symptom cards across request, routing, and streaming failures.