dsh-tool-scheduler

Read-only tool result caching and speculative prefetch for DeepSeek Harness.

An agent reads config.ts. Edits it. Reads it again. Then greps the same directory it just searched.

Each of those is a fresh tool round-trip. On local files it is tens of milliseconds. On the tools that actually hurt — test runners, builds, HTTP fetches, cluster queries — the repeat costs seconds of wall clock. This plugin hooks dsh's tool pipeline and serves those repeats from memory, and pre-reads files it can tell you are about to ask for.

What it skips

A real run of the benchmark suite (3 reads, one of them a repeat) with the plugin enabled:

cache-miss   read  bench/fixture/package.json     execMs 2006
cache-miss   read  bench/fixture/README.md        execMs 2004
cache-hit    read  bench/fixture/package.json     avoidedMs 2006   ← no dispatch

The third read never reached the filesystem. That is the whole idea.

With prefetch on, a glob result feeds the cache ahead of the model:

spec-start   read  bench/fixture/src/add.js
spec-start   read  bench/fixture/src/mul.js
spec-start   read  bench/fixture/src/sub.js
cache-hit    read  bench/fixture/src/add.js    speculative true   ← model asked, nothing ran

Install

dsh plugin --profile web add dsh-tool-scheduler

dsh bundles a default patch so the plugin is active immediately, with caching on and prefetch on. To change settings, override the same entry in your profile:

# ~/.dsh/profiles/web/cordis.patch.yml
- id: tool-scheduler
  config:
    prefetchN: 6
    speculateRootPrefix: "src/"

Requirements: Node 22+, dsh 0.2.x, and pnpm on PATH (dsh's plugin manager drives it directly). On Windows use pnpm 10 — pnpm 12 currently fails its store lock on some setups.

Measured

Same task, plugin off versus on, three runs per cell, median wall time. The slow-tool rows use a benchmark instrument with a fixed 2 s delay per read, so the tool time is known rather than inferred.

Model	Task	Off	On
deepseek-v4.1-flash	3 reads, one repeated	29.9 s	16.0 s	−46%
deepseek-v4.1-flash	glob + 4 reads	15.9 s	14.0 s	−12%
6.8-flash-lite (free tier)	3 reads, one repeated	32.2 s	25.4 s	−21%

Two things worth knowing before you expect these numbers:

The gain tracks model speed. On a fast model the tool share of wall time is larger, so scheduling has more to win. On local millisecond-scale tools it is smaller: in our own attribution runs, local reads and directory listings were 0.1% to 9.4% of wall time, with model latency taking the rest. If your tools are already fast, this plugin will not visibly help you.

The second row is smaller than the first because dsh already runs concurrency-safe tools in parallel batches. Prefetch can only save the window it overlaps, not the whole read.

Method, sample size, axis of measurement and the two results that argue against this plugin are in docs/BENCHMARK.md.

How it works

dsh's tool pipeline is a Cordis waterfall: tools/pre-execute → tools/execute → tools/post-execute. The fallback of tools/execute is the real dispatch. This plugin listens there and, for a whitelisted read-only tool, returns the cached result without calling next() — so the tool body never runs.

Cache keys are the tool name plus its arguments, with path arguments normalized: separators unified and absolute paths resolved back relative to the working directory. That last part matters in practice, because stronger models rewrite relative paths into absolute ones, and without normalization the prefetched entry and the real call land on different keys.

Entries carry a stat fingerprint (size plus mtimeMs) checked on every hit, so a file changed by any means invalidates. bash and pwsh emit no filesystem events in dsh, which is why the stat check exists rather than relying on events alone. fs/write-intent and fs/edit-intent add immediate invalidation for the editor path.

Prefetch runs after tools/post-execute sees a glob result: it takes the first N paths, applies the scope prefix, and calls the same-family read tool with its own abort signal. Those calls go through the same listener, so their results land in the cache through the ordinary miss path.

Configuration

Key	Default
cache	true	Enable read-only result caching
speculate	false (true in the bundled patch)	Enable speculative prefetch
prefetchN	3	Max files prefetched per trigger, 1–64
readonlyTools	[]	Extra tool names to cache
speculateOn	["glob"]	Tools whose results trigger prefetch
speculateRootPrefix	""	Only prefetch under this path prefix; empty means anywhere
validateStat	true	Check the stat fingerprint on every hit
debugLogFile	unset	Write scheduling events here for debugging

Safety

Read-only tools only. bash, pwsh, write and edit are passed through untouched and are never cached or prefetched. Error results, including guard denials, are not remembered. Cache and in-flight prefetches are cleared when the plugin unloads.

Limitations

The cache lives for one process. dsh's headless mode is one process per task, so nothing survives between runs; cross-run persistence is not implemented. The prefetcher is a deterministic rule, not a trained predictor — its hit rate in our benchmark comes from the task shape, and the published PASTE system, which learns patterns from traces, reports only 27.8% top-1 accuracy and leans on multiple candidates. Our n=3 runs are enough to see the first row move and not much else.

Prior art

Speculative tool execution is an active research direction and this package does not claim to have invented it. PASTE predicts and pre-executes future tool calls during generation; TomasuLLM runs tool calls out of order behind an overlay and commits after verification; SPAgent does the equivalent for search agents. This is an installable implementation for DeepSeek Harness, with measurements and a stated applicability boundary.

中文

给 DeepSeek Harness 用的工具调度插件：只读工具的结果会进缓存，重复调用直接命中，不触发真实执行； glob 之后还会提前预读一批文件，等模型真正要的时候直接拿到。

安装：dsh plugin --profile web add dsh-tool-scheduler（需要 Node 22+、dsh 0.2.x、PATH 里有 pnpm， Windows 上用 pnpm 10）。

收益有前提：模型越快、工具越慢，这个插件越有用。本地毫秒级的读文件场景里，工具只占整体耗时的 0.1%–9.4%，其余都是模型延迟，这种情况下基本没有感知。实测数字、方法和反面结果见 docs/BENCHMARK.md。

License

MIT