ChatGPTClaudeGrok / X / CursorGemini

DSH plugin StreamChunk: protocols, failures, and retries

Plugin DevelopmentPublished 2026-11-12Author: DeepSeek Plugin Market
DSH pluginDeepSeek HarnessStreamChunkLlmFailureRetry policy
DSH plugin streaming: seven StreamChunk variants, compact stream replay, LlmFailure versus retry policy, and why one adapter call is one attempt.

When a stream breaks mid-flight, the classic mistake is collapsing "what failed" and "should we retry" into one decision. DSH plugin separates them: LlmFailure describes what happened, and ResolvedRetryPolicy decides whether another attempt is allowed. Mix them, and half-written content gets sent twice.

A streamed response is interleaved blocks, not a paragraph

StreamChunk is a closed discriminated union, and index puts interleaved deltas back in place.(source)

One streamed response can carry text, reasoning, and multiple tool calls at once. index ties each delta to its block, and block-end carries the fully assembled ContentBlock, so consumers never reassemble deltas themselves. Because the union is closed, a switch on type ends with assertNever, and adding a variant breaks compilation at every consumer that must handle it.

The seven variants:

VariantWhat it carries
block-startindex and blockType
text-deltaindex and a text fragment
reasoning-deltaindex and a reasoning fragment
tool-call-deltaindex, call id, optional name, raw argument increment
block-endindex and the assembled block
usagetoken usage
finishend reason and optional replayState

One ordering constraint is hard: usage precedes the terminal finish, and nothing follows finish. Tool arguments stay raw JSON strings the whole way — if a provider hands back parsed objects, the adapter must reserialize at block-end.

The compact stream is replay evidence, not a second message

Compression drops duplicate envelopes; it never merges token boundaries or loses terminal facts.(source)

AssistantStreamAccumulator pairs each StreamChunk with its safe-integer timestamp to produce AssistantStreamRecord[]. Consecutive text, reasoning, or tool-argument deltas in the same block collapse into one record: time0, exact timestamp intervals, and one array member per original delta; other chunks stay as timestamped raw records.

That compression has three "never"s: never merge token boundaries, never drop terminal, usage, block, failure, or replay facts, and never reorder. So it shrinks the log substantially while still reconstructing the exact timed stream on replay.

snapshot() returns a detached, immutable stream; expandAssistantStream() strictly validates record keys, member counts, indexes, timestamps, tool-call identity, and lossless JSON before rebuilding the exact sequence. The session log embeds the stream into the assistant/message that serves as the surface result, or into an assistant/attempt when no surface message exists.

The process-local agent/assistant-stream frame carries live presentation only and never enters the durable feed. Record-level readers such as assistantStreamFirstTokenTime, assistantStreamHasVisibleText, and lastAssistantStreamChunk exit early inside a single scan, so settling a large history costs O(records), not O(members).

LlmFailure is a fact, not a retry verdict

It describes the failure only; whether to retry lives elsewhere.(source)

ts
interface LlmFailure {
  readonly message: string
  readonly code: string
  readonly status?: number
  readonly providerRetryAfterMs?: number
  readonly requestId?: ProviderRequestId
  readonly offloadImages?: number
}

code is a stable, provider-neutral machine routing code; route on it and never on provider prose. status is the HTTP status when available. providerRetryAfterMs is a validated, provider-requested positive delay, and the reference is explicit that it is not a retry decision — a policy may honor or ignore it. requestId is an opaque branded string for diagnostics.

offloadImages only accompanies IMAGE_OFFLOAD_REQUIRED: it tells routing how many of the oldest retained images must be offloaded before the same request fits exact byte metering, and dsh-compaction-image-offload records the chosen occurrence and retries the step.

Two error paths, one failure type

Thrown failures and in-band failures converge on the same payload.(source)

Both supported paths share LlmFailure: throwing from stream() for transport or protocol errors, or ending the stream with finish { kind: 'error' | 'aborted', failure } for adapters that cannot throw mid-stream when the provider reports an in-band error. LlmError.failure carries the same type.

After the selected adapter is invoked, the stream retains the exact thrown Error object and associates the immutable facts plus the immutable retry policy of the actually serving registration with that call. The agent loop's order is fixed:

  1. Commit the attempt stream as assistant/attempt;
  2. Close the failed step;
  3. Hand the error, facts, immutable prior-retried failure facts, actual service policy, and turn signal to agent/request-error;
  4. Only a listener returning { kind: 'retry' } after awaited repair takes over recovery.

Without recovery, the structured failure becomes a turn error, and that attempt commits no surface Assistant message and no tool side effects. Committing before closing is precisely what stops half-written content from being mistaken for a successful message.

Four adapter rules that decide duplicate output

These four decide whether a retry repeats content.(source)

  1. One adapter call is one provider attempt — adapters must disable library-level retries. Agent-layer recovery opens another persisted, numbered turn; callers invoking ctx.llm.stream() directly still get a single attempt.
  2. Stalls are bounded by transport — both delivered remote adapters expose a positive finite streamIdleTimeoutMs, defaulting to five minutes. The watchdog starts only while the iterator's next() is pending, the whole request shares one stable signal, its own expiry maps to TIMEOUT, and an earlier caller abort stays ABORTED.
  3. Context overflow has exactly one canonical code — both DeepSeek adapters classify through isContextWindowExceededError() and expose CONTEXT_WINDOW_EXCEEDED, whether the failure arrives as a thrown HTTP LlmError or an in-band finish error.
  4. Empty completion is retryable — a terminal stop carrying no content blocks maps to finish { kind: 'error' } with EMPTY_RESPONSE, which dsh-llm-retry retries by default.

ResolvedRetryPolicy: frozen after registration

Policies resolve into an immutable union before route registration, so in-flight failures ignore later swaps.(source)

Two modes:

  • normal — finite maxRetries, retryableCodes, and required initialDelayMs, maxDelayMs, and jitterRatio;
  • always — mode: 'always' with the same required backoff fields but no finite cap.

Omitting a provider policy uses a normal default with five retries. After layered settings switch from normal to always, leftover maxRetries or retryableCodes fields may remain; the resolver ignores disabled fields and captures a pure always policy.

There are two read entry points: LlmRuntime.providerRetryPolicy(provider) returns the registered value, and llmRetryPolicyOf(stream) returns the value captured from the registration that actually served the call. The latter is how you answer "why did this attempt not retry" — releasing or replacing a route afterwards cannot change the recovery policy of a failure already in flight.

Verification: inject a mid-stream break

Retry verification checks two things: correct classification and no duplicated output.(source)

  1. Inject an adapter that throws after a text-delta and assert the error is normalized into a terminal finish, not a bare exception.
  2. Assert the attempt commits as assistant/attempt with no surface Assistant message.
  3. Return { kind: 'retry' } from agent/request-error and assert the recovery turn is a new persisted number rather than the old attempt.
  4. Let the retry succeed and assert history holds exactly one assistant message without the first half-written text.
  5. Build a response with a stop but no content blocks and assert the failure code is EMPTY_RESPONSE and the default policy retries it.
  6. Force context overflow and assert CONTEXT_WINDOW_EXCEEDED, not a match on provider prose.

If step 4 yields two messages, the adapter probably retried at the library level, violating one-call-one-attempt.

For stream stalls and gzip buffering, see gzip stream buffering fix; for adapter routing, see LLM adapters.

One line to close: chunks are an interleaved block protocol, compact streams squeeze envelopes but not facts, failures record facts while policies decide retries, and one adapter call is one attempt.

ChatGPTClaudeGrok / X / CursorGemini

FAQ

Must a DSH plugin consumer assemble StreamChunk blocks?

No. A DSH plugin block-end carries the fully assembled ContentBlock. DSH plugin uses index to associate interleaved deltas, so consumers handle the block-end result and never reassemble.

Is providerRetryAfterMs a retry decision in DSH plugin?

No. DSH plugin treats it as a provider-requested delay only. DSH plugin's LlmFailure carries serializable facts, while the immutable retry policy decides whether another attempt happens.

May a DSH plugin adapter do library-level retries?

No. DSH plugin defines one adapter call as one provider attempt. DSH plugin requires adapters to disable library retries; recovery happens when the agent layer opens another persisted, numbered turn.

Is an empty completion a success in DSH plugin?

No. DSH plugin maps it to a retryable error. DSH plugin adapters map a terminal stop with no content blocks to an error finish with EMPTY_RESPONSE, which the default policy retries.

Related Terms

StreamChunk
StreamChunk is the raw streaming protocol emitted by adapters: a closed discriminated union whose block-end carries the assembled ContentBlock.— packages/llm/llm/src/types.ts
AssistantStreamRecord
AssistantStreamRecord compacts an assistant stream by pairing chunks with safe-integer timestamps while preserving delta boundaries.— DeepSeek Harness reference - LLM streaming
LlmFailure
LlmFailure is a serializable, provider-neutral failure fact: message, code, optional status, providerRetryAfterMs, and requestId.— packages/llm/llm/src/types.ts
ResolvedRetryPolicy
ResolvedRetryPolicy is the immutable retry policy resolved before route registration, with normal and always modes.— DeepSeek Harness reference - LLM streaming

Sources