@richie.liu/dsh-hybrid-coder

English | 中文

Dual-model routing policy plugin: a premium model plans and rescues, a local small model (e.g. a local Ollama instance) implements ordinary steps, and the route escalates back to premium automatically after consecutive local failures.

This plugin is a routing policy, not a model transport: model requests still go out through the registered LLM adapters (e.g. @deepseek-ai/dsh-llm-deepseek, @deepseek-ai/dsh-llm-pi-ai). It rewrites the target provider/model at each step's request assembly point and switches routes when the tool-failure chain crosses a threshold. Experimental status: public contracts may change and it is not shipped with official releases.

Installation

After publishing to npm, install it in the target profile with one command (the package's cordis.patch.yml declares dsh.bundle, so installing also activates it as a profile layer):

dsh plugin --profile web add @richie.liu/dsh-hybrid-coder

Or enable it via a local tarball / overlay while developing (see "Local provider configuration (Ollama)" below). The bundled example config points at GLM + Ollama routes; override premium/local provider and model to match your environment.

How it works

A "step" is one model request plus the tool calls it triggers. While each step assembles its request, the plugin decides the route by the following priority:

1. Escalated → premium. The escalation latch set by a previous local failure streak has not been released.
2. Plan mode active → premium. Planning always uses the stronger model.
3. Otherwise → local.

Plan-mode state folds directly from the plan/mode events in the session log (@deepseek-ai/dsh-plan-mode); the plugin stores no planning state of its own. The escalation latch and success counts fold from the log too, so fork, resume, and process restart recover identical routing decisions with no in-process live state.

Escalation (Strategy B)

A tools/post-execute listener observes every tool execution result. While the effective provider is local, it counts "consecutive failed tool executions":

• Failure = the tool-result block's isError: true on tool/result (the always-persisted authoritative failure signal). The error field exists only when the tool threw a machine-coded HarnessError; failures thrown as plain Error count the same.
• Cancelled results whose error.code is ABORTED or ABORTED_BEFORE_DISPATCH are excluded (cancellation is not a model capability problem).
• The count resets on any non-error tool result, on a new turn (turn/start), and when the effective provider leaves local.
• The window is the current turn: each new user turn gives the local model a fresh chance.

When the consecutive failure count within one turn reaches failureThreshold:

1. Append the durable event hybrid/route { to: 'premium', reason: 'tool-failures', turn, step };
2. Inject an escalation guidance message into the next step's inbox (see below), containing the recent failure trajectory (bounded);
3. Later agent/request folds to that event and routes to premium.

Request-level fallback

An agent/request-error listener handles transport-level failures of the local provider (failure.code is TRANSPORT or TIMEOUT, e.g. Ollama not running, connection refused). It first appends hybrid/route { to: 'premium', reason: 'request-failure' }, then returns { kind: 'retry' } so the loop re-issues the same step with premium; other error codes are left to @deepseek-ai/dsh-llm-retry.

De-escalation

After escalation, the plugin counts "clean premium steps": a step whose effective provider is premium, which produced an assistant message, and had no error tool results (a plain text reply counts as success). When the count reaches premiumStepsBeforeDeescalation, it appends hybrid/route { to: 'local', reason: 'recovered' } and routing returns to local. De-escalation injects no guidance message.

System-prompt identity sync

The product system prompt contains identity variables such as "powered by the {{model}} model", whose defaults come from the declared route rather than the per-step request config. Without handling, a request switched to local would still claim to be the premium model. The plugin therefore also listens to system-prompt/assemble and overwrites the provider/model template variables with the next step's actual route, keeping identity consistent with model selection.

When a request-level fallback flips the route mid-step, that retry step's identity text may still refer to the previous route; this is a benign directional skew (the stronger premium model is actually serving) and is recorded under Known Limitations.

Config

- id: hybrid-coder
  name: '@richie.liu/dsh-hybrid-coder'
  config:
    premium:
      provider: glm
      model: glm-4-plus
      reasoningEffort: high        # optional; unset keeps the provider default
    local:
      provider: ollama
      model: qwen2.5-coder:7b
    escalation:
      failureThreshold: 2                  # consecutive failed tool executions in the current turn, >= 1
      premiumStepsBeforeDeescalation: 2    # consecutive clean premium steps, >= 1

Unknown config keys fail at load. premium.provider and local.provider must be registered provider routes (see ctx.llm.listProviders()); when the target route is missing at the first routing decision, the request fails with an explicit error instead of silently falling back.

reasoningEffort applies to the premium route only; the local route always clears any inherited effort and restores its provider's default behavior.

Local provider setup (Ollama)

The local model connects through @deepseek-ai/dsh-llm-pi-ai's hand-declared routes, no code needed:

- id: llm-pi-ai
  name: '@deepseek-ai/dsh-llm-pi-ai'
  config:
    providers:
      ollama:
        displayName: Ollama (local)
        api: openai-completions
        apiKeyEnv: OLLAMA_API_KEY
        baseURL: http://localhost:11434/v1
        models:
          - id: qwen3:4b-32k
            name: Qwen3 4B
            contextWindow: 32768
        retryPolicy:
          mode: normal
          maxRetries: 0

Three settings proved necessary in live runs (all three required):

• apiKeyEnv must be declared: pi-ai's openai-completions protocol requires a credential reference to exist, or the request fails with PI_AI_ERROR: No API key for provider. Ollama ignores the bearer value; a placeholder environment variable (e.g. OLLAMA_API_KEY=ollama) suffices.
• Context length must be >= 32768: Ollama's default num_ctx is 4096, which cannot hold the harness system prompt plus tool definitions (~13000 tokens measured) — the model never sees the tools and replies in plain text. The profile's contextWindow is metadata only and does not change Ollama behavior; derive a model via a Modelfile: printf 'FROM qwen3:4b\nPARAMETER num_ctx 32768\n' > Modelfile && ollama create qwen3:4b-32k -f Modelfile.
• The model must return structured tool_calls: in live testing qwen2.5-coder:7b (which declares tools support) emitted tool calls as plain-text JSON on Ollama's OpenAI-compatible endpoint (tool_calls: null), so the turn ended as plain text; qwen3:4b returns structured calls and works. Verify before adopting a model.

Premium provider setup (GLM)

The premium route is declared the same way — another hand-declared provider on the same adapter, pointing at an OpenAI-compatible endpoint:

      glm:
        displayName: Zhipu GLM
        api: openai-completions
        apiKeyEnv: GLM_API_KEY
        baseURL: https://open.bigmodel.cn/api/paas/v4
        models:
          - id: glm-4-plus
            name: GLM-4-Plus
            contextWindow: 128000
        retryPolicy:
          mode: normal
          maxRetries: 0

Then point premium.provider / premium.model at it in the hybrid-coder config (e.g. glm / glm-4-plus). The key comes from the GLM_API_KEY environment variable or ~/.dsh/.credentials.yaml; retryPolicy.maxRetries: 0 for the same reason as local — let the plugin own transport failover immediately.

Composition contract with llm-retry

@deepseek-ai/dsh-llm-retry registers by default on the outer layer of the request-error recovery chain. A TRANSPORT from a dead Ollama endpoint belongs to the default retryable codes; with the default retryPolicy (5 backoff attempts) on the local route, llm-retry would back off against the dead endpoint for roughly 5 rounds before this plugin gets to switch to premium — a slow failover.

The local route must therefore set retryPolicy.maxRetries to 0 (or remove TRANSPORT from retryableCodes), so llm-retry delegates immediately and this plugin owns transport failover instantly. This is provider-owned configuration (retryPolicy belongs to each provider config, not this plugin's config), consistent with the architecture's "providers own retryPolicy" separation of concerns.

Durable events

The plugin adds one event to SessionEventMap:

hybrid/route { to: 'premium' | 'local', reason: 'tool-failures' | 'request-failure' | 'recovered', turn: number, step: number }

• tool-failures: local consecutive tool failures reached the threshold; escalate.
• request-failure: local request failed at transport level; escalate immediately and retry.
• recovered: enough clean premium steps after escalation; de-escalate back to local.

turn/step name the turn and step open when the event fires. The event records only the sticky escalation latch; premium routing caused by plan mode is not re-persisted (re-derived from plan/mode every step). The event is registered through the persistence catalog generator and persists with the session log across fork and resume.

The ./invariant companion plugin replays and validates before each append: payload shape, turn/step ownership and monotonicity, and legal transitions (to: 'premium' may only enter from a non-escalated state; to: 'local' may only recover from an escalated state).

Model Experience

Escalation guidance

What the model sees

When a tool-failure streak triggers escalation, the next step's model receives a user-role message whose source is a plugin notification (source.kind: 'plugin'), containing fixed framework text plus the recent failure trajectory. The framework text is verbatim:

The previous model made repeated failed tool calls. A stronger model is now handling the session. Diagnose the failure from the trajectory below and continue the task. Do not repeat the failing approach.

Recent failed tool calls:

The failed tool names and error messages follow, one bullet each. The trajectory keeps only the last 4 entries, each error message truncated; the whole message is capped at 2000 UTF-8 bytes. Older failure entries are dropped first when over budget. With no failure trajectory the message does not appear (which cannot happen, since escalation is itself triggered by failures).

Token effects

The message appends exactly once per escalation trigger, as conditional, bounded (≤2000 bytes), append-only input. Plan-mode routing, de-escalation, and request-level fallback add no model tokens by themselves.

KV cache effects

The escalation flip replaces the request's provider/model prefix, invalidating cache reuse for that provider (the model endpoint physically switches); the premium steps after escalation share a stable prefix, and de-escalation to local behaves the same. The guidance message appends after the reusable history and does not change the previously persisted prefix.

Known Limitations and Deferred Work

• No AST skeletonized context: file contents enter the model only as tool results after the model explicitly calls a reading tool; the context assembly stage does not mount raw file content, so the originally envisioned "pre-step skeletonized files" has no object to act on. Oversized tool results are handled by @deepseek-ai/dsh-spill-policy. A future tools/post-execute step could replace read results for the local route with signature skeletons (requires a TypeScript compiler dependency and full syntax-surface coverage); not implemented today.
• No automatic file-write rollback: the plugin owns no filesystem transactions; erroneous writes from the local model are corrected through normal tool-result feedback and the escalation flow, not by undoing disk changes automatically.
• Routing overrides user model choice: mounting this plugin means it owns routing for every agent in the composition; explicit per-session model selections are kept only when they match the configured route pair. There is no opt-out switch to honor explicit selections (no evidence of a current consumer).
• Retry-step identity text may lag: a request-level fallback flips the route mid-step, so that retry step's system-prompt identity variables may still show local while premium is actually serving. Benign directional skew; does not affect results.
• Experimental contract: event names, config fields, and guidance text may change before the first tagged release; persisted logs promise no cross-version compatibility (consistent with the repository's pre-release stance).