DSH plugin StreamChunk: protocols, failures, and retries
When a stream breaks mid-flight, the classic mistake is collapsing "what failed" and "should we retry" into one decision. DSH plugin separates them: LlmFailure describes what happened, and ResolvedRetryPolicy decides whether another attempt is allowed. Mix them, and half-written content gets sent twice.
A streamed response is interleaved blocks, not a paragraph
StreamChunk is a closed discriminated union, and index puts interleaved deltas back in place.(source)
One streamed response can carry text, reasoning, and multiple tool calls at once. index ties each delta to its block, and block-end carries the fully assembled ContentBlock, so consumers never reassemble deltas themselves. Because the union is closed, a switch on type ends with assertNever, and adding a variant breaks compilation at every consumer that must handle it.
The seven variants:
| Variant | What it carries |
|---|---|
block-start | index and blockType |
text-delta | index and a text fragment |
reasoning-delta | index and a reasoning fragment |
tool-call-delta | index, call id, optional name, raw argument increment |
block-end | index and the assembled block |
usage | token usage |
finish | end reason and optional replayState |
One ordering constraint is hard: usage precedes the terminal finish, and nothing follows finish. Tool arguments stay raw JSON strings the whole way — if a provider hands back parsed objects, the adapter must reserialize at block-end.
The compact stream is replay evidence, not a second message
Compression drops duplicate envelopes; it never merges token boundaries or loses terminal facts.(source)
AssistantStreamAccumulator pairs each StreamChunk with its safe-integer timestamp to produce AssistantStreamRecord[]. Consecutive text, reasoning, or tool-argument deltas in the same block collapse into one record: time0, exact timestamp intervals, and one array member per original delta; other chunks stay as timestamped raw records.
That compression has three "never"s: never merge token boundaries, never drop terminal, usage, block, failure, or replay facts, and never reorder. So it shrinks the log substantially while still reconstructing the exact timed stream on replay.
snapshot() returns a detached, immutable stream; expandAssistantStream() strictly validates record keys, member counts, indexes, timestamps, tool-call identity, and lossless JSON before rebuilding the exact sequence. The session log embeds the stream into the assistant/message that serves as the surface result, or into an assistant/attempt when no surface message exists.
The process-local agent/assistant-stream frame carries live presentation only and never enters the durable feed. Record-level readers such as assistantStreamFirstTokenTime, assistantStreamHasVisibleText, and lastAssistantStreamChunk exit early inside a single scan, so settling a large history costs O(records), not O(members).
LlmFailure is a fact, not a retry verdict
It describes the failure only; whether to retry lives elsewhere.(source)
interface LlmFailure {
readonly message: string
readonly code: string
readonly status?: number
readonly providerRetryAfterMs?: number
readonly requestId?: ProviderRequestId
readonly offloadImages?: number
}
code is a stable, provider-neutral machine routing code; route on it and never on provider prose. status is the HTTP status when available. providerRetryAfterMs is a validated, provider-requested positive delay, and the reference is explicit that it is not a retry decision — a policy may honor or ignore it. requestId is an opaque branded string for diagnostics.
offloadImages only accompanies IMAGE_OFFLOAD_REQUIRED: it tells routing how many of the oldest retained images must be offloaded before the same request fits exact byte metering, and dsh-compaction-image-offload records the chosen occurrence and retries the step.
Two error paths, one failure type
Thrown failures and in-band failures converge on the same payload.(source)
Both supported paths share LlmFailure: throwing from stream() for transport or protocol errors, or ending the stream with finish { kind: 'error' | 'aborted', failure } for adapters that cannot throw mid-stream when the provider reports an in-band error. LlmError.failure carries the same type.
After the selected adapter is invoked, the stream retains the exact thrown Error object and associates the immutable facts plus the immutable retry policy of the actually serving registration with that call. The agent loop's order is fixed:
- Commit the attempt stream as
assistant/attempt; - Close the failed step;
- Hand the error, facts, immutable prior-retried failure facts, actual service policy, and turn signal to
agent/request-error; - Only a listener returning
{ kind: 'retry' }after awaited repair takes over recovery.
Without recovery, the structured failure becomes a turn error, and that attempt commits no surface Assistant message and no tool side effects. Committing before closing is precisely what stops half-written content from being mistaken for a successful message.
Four adapter rules that decide duplicate output
These four decide whether a retry repeats content.(source)
- One adapter call is one provider attempt — adapters must disable library-level retries. Agent-layer recovery opens another persisted, numbered turn; callers invoking
ctx.llm.stream()directly still get a single attempt. - Stalls are bounded by transport — both delivered remote adapters expose a positive finite
streamIdleTimeoutMs, defaulting to five minutes. The watchdog starts only while the iterator'snext()is pending, the whole request shares one stable signal, its own expiry maps toTIMEOUT, and an earlier caller abort staysABORTED. - Context overflow has exactly one canonical code — both DeepSeek adapters classify through
isContextWindowExceededError()and exposeCONTEXT_WINDOW_EXCEEDED, whether the failure arrives as a thrown HTTPLlmErroror an in-band finish error. - Empty completion is retryable — a terminal stop carrying no content blocks maps to
finish { kind: 'error' }withEMPTY_RESPONSE, whichdsh-llm-retryretries by default.
ResolvedRetryPolicy: frozen after registration
Policies resolve into an immutable union before route registration, so in-flight failures ignore later swaps.(source)
Two modes:
normal— finitemaxRetries,retryableCodes, and requiredinitialDelayMs,maxDelayMs, andjitterRatio;always—mode: 'always'with the same required backoff fields but no finite cap.
Omitting a provider policy uses a normal default with five retries. After layered settings switch from normal to always, leftover maxRetries or retryableCodes fields may remain; the resolver ignores disabled fields and captures a pure always policy.
There are two read entry points: LlmRuntime.providerRetryPolicy(provider) returns the registered value, and llmRetryPolicyOf(stream) returns the value captured from the registration that actually served the call. The latter is how you answer "why did this attempt not retry" — releasing or replacing a route afterwards cannot change the recovery policy of a failure already in flight.
Verification: inject a mid-stream break
Retry verification checks two things: correct classification and no duplicated output.(source)
- Inject an adapter that throws after a
text-deltaand assert the error is normalized into a terminalfinish, not a bare exception. - Assert the attempt commits as
assistant/attemptwith no surface Assistant message. - Return
{ kind: 'retry' }fromagent/request-errorand assert the recovery turn is a new persisted number rather than the old attempt. - Let the retry succeed and assert history holds exactly one assistant message without the first half-written text.
- Build a response with a stop but no content blocks and assert the failure code is
EMPTY_RESPONSEand the default policy retries it. - Force context overflow and assert
CONTEXT_WINDOW_EXCEEDED, not a match on provider prose.
If step 4 yields two messages, the adapter probably retried at the library level, violating one-call-one-attempt.
For stream stalls and gzip buffering, see gzip stream buffering fix; for adapter routing, see LLM adapters.
One line to close: chunks are an interleaved block protocol, compact streams squeeze envelopes but not facts, failures record facts while policies decide retries, and one adapter call is one attempt.
FAQ
No. A DSH plugin block-end carries the fully assembled ContentBlock. DSH plugin uses index to associate interleaved deltas, so consumers handle the block-end result and never reassemble.
No. DSH plugin treats it as a provider-requested delay only. DSH plugin's LlmFailure carries serializable facts, while the immutable retry policy decides whether another attempt happens.
No. DSH plugin defines one adapter call as one provider attempt. DSH plugin requires adapters to disable library retries; recovery happens when the agent layer opens another persisted, numbered turn.
No. DSH plugin maps it to a retryable error. DSH plugin adapters map a terminal stop with no content blocks to an error finish with EMPTY_RESPONSE, which the default policy retries.
Related Terms
- StreamChunk
- StreamChunk is the raw streaming protocol emitted by adapters: a closed discriminated union whose block-end carries the assembled ContentBlock.— packages/llm/llm/src/types.ts
- AssistantStreamRecord
- AssistantStreamRecord compacts an assistant stream by pairing chunks with safe-integer timestamps while preserving delta boundaries.— DeepSeek Harness reference - LLM streaming
- LlmFailure
- LlmFailure is a serializable, provider-neutral failure fact: message, code, optional status, providerRetryAfterMs, and requestId.— packages/llm/llm/src/types.ts
- ResolvedRetryPolicy
- ResolvedRetryPolicy is the immutable retry policy resolved before route registration, with normal and always modes.— DeepSeek Harness reference - LLM streaming
Sources
- DeepSeek Harness reference - LLM streaming· deepseek-ai
- packages/llm/llm/src/types.ts· deepseek-ai