dsh-retry-after

Stop guessing the backoff: let the provider tell you how long to wait.

A plugin for DeepSeek Harness that bridges the HTTP Retry-After header into the harness retry policy — on the routes where it is currently thrown away.

中文说明 · Design notes

---

The problem

When a gateway answers 429 Too Many Requests, it usually says when to come back:

HTTP/1.1 429 Too Many Requests
retry-after: 20

The harness already knows how to honour that: LlmFailure.providerRetryAfterMs exists exactly for it, dsh-llm-retry reads it, and the native DeepSeek transport fills it in from the response headers.

Every OpenAI-compatible route (the llm-pi-ai plugin, which covers OpenCode, OpenRouter, vLLM, LiteLLM and friends) loses it on the way:

1. @earendil-works/pi-ai catches the provider error and rethrows new Error(...) — status and headers gone.
2. dsh-llm-pi-ai classifies that string with classifyPiAiError, which pattern-matches the message text.
3. dsh-llm-retry gets a failure with no timing hint, so it falls back to its own backoff: initialDelayMs (500 ms by default), doubling, jittered.

So a gateway that asks for 20 seconds gets hit again after 500 milliseconds. Retries burn double, and a soft rate limit turns into a hard one.

 provider says  retry-after: 20
        │
        ▼
   pi-ai SDK ──── Error("429 ...") ────► headers dropped ✂
        │
        ▼
 dsh-llm-retry ── no hint ──► waits 500ms, 1000ms, 2000ms ...

What this plugin does

It reads the headers before they are dropped — at the fetch edge — and writes them back onto the failure object, in the one slot dsh-llm-retry already reads.

   fetch edge ──► capture (status, retry-after) ──► ring buffer
                                                       │
 dsh-agent-loop ── agent/request-error ──► prepend listener ──► failure.providerRetryAfterMs
                                                       │
                                              dsh-llm-retry waits exactly that long

Concretely, on every agent/request-error it:

Fact	Added as	Why
response status	failure.status	diagnostics; keeps the record honest
Retry-After (or Retry-After-Ms)	failure.providerRetryAfterMs	the delay the provider asked for
corrected error code	failure.code	a localized 429 body ("请求过于频繁") classifies as PI_AI_ERROR and never enters the retry list; the captured status does not care what language the gateway writes in

Nothing else changes. The retry budget, jitter, cancellation, llm/retry events and per-provider retryPolicy all stay with dsh-llm-retry. This is a translator, not a second retry loop.

Measured effect

A mock gateway that answers 429 with retry-after: 2 three times, then succeeds (test/mock-gateway.mjs, test/mock.patch.yml):

	attempt gap #1→#2	#2→#3
without the plugin	500 ms	1000 ms (local backoff)
with the plugin	2030 ms	2030 ms (what the gateway asked for)

Reproduce it yourself:

node test/mock-gateway.mjs &                 # 429 + retry-after: 2, three times
DSH_HOME=/tmp/dsh-retry-after-demo \
  dsh --profile demo --patch ./test/mock.patch.yml headless "say hi"

Install

dsh plugin --profile tui add dsh-retry-after

Then restart the profile. The bundle patch registers the retry-after row automatically; there is nothing else to configure.

Until the package is on npm, install straight from the repository:

dsh plugin --profile tui add github:PandaAQI/dsh-retry-after

Verify it is composed (the row carries your config):

dsh --profile tui --dump-config | grep -A6 retry-after

Configuration

Everything is optional; the defaults are what you want almost always. The row lives in your profile's cordis.patch.yml:

- id: retry-after
  config:
    windowMs: 15000        # how long a captured response stays matchable
    reclassify: true       # fix PI_AI_ERROR/UNKNOWN using the captured status
    oversize: clamp        # what to do when the provider asks for more than maxDelayMs
    matchUnmatched: true   # let a provider without a baseURL borrow a lone failing origin
    floorMsByCode: {}      # e.g. { RATE_LIMIT: 5000 } for 429s with no header
    debug: false           # log every annotation

oversize

dsh-llm-retry gives up instead of retrying when the requested delay exceeds retryPolicy.maxDelayMs (in normal mode). That is a reasonable default — the provider just told you it is busy for an hour — but it can also kill a turn that would have recovered after a slightly longer wait.

• clamp (default) — wait maxDelayMs, keep retrying. Never wait longer than your policy allows.
• honor — pass the provider's number through. An oversized hint ends the retry, as designed upstream.
• ignore — drop the hint, use local backoff. Useful when the gateway's numbers are nonsense.

floorMsByCode

Risk-control gateways sometimes answer 429 with no Retry-After at all. A floor gives those a minimum wait, so local backoff cannot hammer them:

- id: retry-after
  config:
    floorMsByCode:
      RATE_LIMIT: 5000

Limitations

Read these before relying on it.

• Correlation is a heuristic. pi-ai hands the harness no request id, so a failure is matched to a capture by origin (scheme + host + port) and recency (windowMs). Two providers sharing one origin, failing within the window, can be attributed to each other. With one provider per origin — the normal case — this is exact.
• Concurrency. Parallel subagent requests to the same origin share the ring buffer; the newest record wins. A 429 for one request may annotate another. The delay is still a real number from that origin, so the cost is a suboptimal wait, not a wrong retry.
• The failure object is frozen. This was a surprise during development: Object.isExtensible is false on the failure the agent loop hands to the waterfall, so writes throw. The plugin clones it and replaces payload.failure, which works because dsh-llm-retry destructures the payload after this prepended listener runs. If a future harness freezes the payload too, the plugin degrades to doing nothing (and says so in debug).
• Fetch-based transports only. Routes using the WebSocket transport never produce an HTTP response to observe; they are untouched (they also carry their own error handling).
• It does not reduce burst. Honouring Retry-After makes each retry wait longer; it does not stop the request that caused the 429. Pair it with sane prompt/context sizes if you are hitting hard limits repeatedly.
• Native routes do not need it. deepseek-official already fills providerRetryAfterMs; the plugin is a no-op there (it finds no capture and leaves the failure alone).

When to use something else

• You want more retries with a different policy → dsh-retry-boost
• You want to fail over to another model or provider → dsh-llm-fallbacks
• You want the transport fixed upstream → @earendil-works/pi-ai #9595, #8507 (both declined); the harness-side ask is discussion #8992

Development

node --test test/          # unit tests (node:test, no deps)
node test/mock-gateway.mjs # mock gateway for manual runs

Layout:

lib/capture.js    fetch observer, Retry-After parsing, ring buffer
lib/annotate.js   provider→origin resolution, failure annotation
lib/index.js      plugin entry, config validation, wiring
test/             unit tests + mock gateway + verification overlay
docs/DESIGN.md    the source lines this relies on

License

MIT