dsh-verified-progress

A verified-progress ledger for long-running DSH work — where a verdict only counts if the thing that produced it provably stayed read-only.

A long-running task fails in a specific way: the agent reports progress it cannot prove, the report enters the transcript, and the next turn builds on a claim that was never checked. Nothing downstream can tell a verified step from an optimistic one, so the error compounds until the final answer is confidently wrong.

This plugin adds the missing check. When the agent claims a step is done, an independent verifier with a fresh context inspects the real workspace and returns a structured verdict. Only a clean, aligned, complete verdict enters the ledger. A rejected step stays in the ledger as evidence — it is never promoted to progress.

claim ──▶ independent verification ──▶ verdict
                                        │
                        complete + clean + aligned ──▶ ledger (progress)
                                        │
                                    anything else ──▶ ledger (evidence only)

The part that is not a prompt

Read-only verifiers are usually enforced by asking the verifier to behave and by removing the write tools from its tool set. Both are prevention, and both are defeated by anything the tool filter does not name.

This plugin detects instead. The workspace is fingerprinted — every file's size and mtime, plus a SHA-256 for files under the hash limit — immediately before and after each verification episode. If anything changed, the verifier mutated the workspace during a read-only audit, so its own findings are voided by construction: the round is recorded as blocked / violation and can never be promoted to progress, no matter how clean its report reads.

That matters because a verifier that writes is not a neutral witness. It can create the evidence it then reports, and no amount of prompt discipline or tool allow-listing proves it did not. A fingerprint comparison does not have to trust the verifier at all — which is why the ledger's admission rule can be absolute:

integrity === "violation" || contract !== "aligned"  ⟹  status !== "complete"

This is enforced in code, not requested in a prompt. A verifier that omits a control line degrades conservatively — an unstated integrity is read as suspect, never as clean — because a placeholder must never be the thing that certifies work as done.

What it does

1. Independent verification instead of self-report. The verifier is a separate subagent that inherits no conversation context. It reads the files and answers with three control lines:

Status: complete | incomplete | blocked
Integrity: clean | suspect | violation
Contract audit: aligned | unknown | needs_revision | invalid

2. A durable verified-progress ledger. Accepted steps are appended to ledger.jsonl (one JSON object per line, fsync'd per append) with a last-wins projection in state.json. A crashed process, a compacted context, or a fresh session can reopen the run and answer: what is verified, what was rejected and why, and what is left. Rejected rounds are preserved as evidence rather than discarded, so a later turn can see which claims were already tried and failed.

3. Mutation detection. Described above; it is the admission rule for the ledger, not a side feature.

Tools

Tool	Purpose
verified_progress_verify	Verify one claimed step against the workspace; records the verdict in the ledger.
verified_progress_ledger	Read back verified progress, rejected claims, and what remains.
verified_progress_state	Current run state: verified rounds, pending claims, open items.

Declaring what the guard may judge

verified_progress_verify takes a guard_scope: a workspace-relative path or glob whose non-glob prefix becomes the fingerprint root.

verified_progress_verify  claim="the parser handles CRLF input"
                          guard_scope="src/parser/**"

This is not optional decoration. The guard originally fingerprinted the whole workspace root, and in the desktop profile that root is the harness home itself — sessions/**, storages/** and plugin state are written continuously by the very session the plugin runs inside. The first two verifications came back workspace mutated (+3 ~5 -3), and every changed path belonged to the harness. The verifier had written nothing. A guard that accuses the verifier of the host's writes fails every round while looking like a conservative verdict, so the scope is now declared rather than assumed:

• the scope must resolve inside the workspace; absolute paths and .. are rejected instead of silently widened;
• when the workspace is the harness home, a scope is required, and harness-owned roots (sessions, storages, logs, telemetry, cache, profiles, …) are excluded only at the root, so a real project containing a nested sessions/ is still guarded;
• the result prints Fingerprinted: <path>, so a verdict never leaves its scope ambiguous.

Install

dsh plugin --profile web add dsh-verified-progress

Or install a released tarball, which needs no registry:

dsh plugin --profile web add \
  https://github.com/lion231226/dsh-verified-progress/releases/download/v0.3.1/dsh-verified-progress.tgz

Then restart the harness: a plugin that failed to activate is not retried, and the module cache holds the previous version until the process restarts. No build step — the package ships plain ESM and has no runtime dependencies.

Storage

Everything lives under the harness state directory, not in your project:

<state>/verified-progress/runs/<runId>/ledger.jsonl
<state>/verified-progress/runs/<runId>/state.json

<runId> is derived from the session id when there is one, otherwise from a hash of the working directory, so reopening the same session reopens the same run.

Design limits — read these

• The verifier is a separate agent, not a separate process. It cannot see the executor's conversation, which is what makes it independent, and it runs with a read-only tool set. It runs inside the harness and is subject to the same permissions; the fingerprint check is what makes a violation detectable rather than impossible.
• The mutation guard is a correctness tool, not a security boundary. It compares file fingerprints. It does not defend against a local attacker, it is not a sandbox, and it does not cover paths excluded from the snapshot.
• Unhashed large files are reported as an evidence gap. Files above the hash limit are compared by size and mtime, and the snapshot says so rather than claiming a comparison it did not make.
• Verification costs tokens. Each verified step adds one verifier episode. The ledger is what you get for it; if your task is a single short turn, you do not need this plugin.

Why this exists

The loop design is ported from LongHorizon-Harness (AMAP-ML, MIT) — manager/executor/auditor rounds, a durable round ledger, and an auditor that cannot certify a dirty result.

That project is a Python orchestrator that wraps an agent CLI. It does not run on Windows: its persistent layer requires os.O_NOFOLLOW, os.O_DIRECTORY and supports_dir_fd (all absent on win32) and it shells out with POSIX VAR=value cmd templates. It also, by its own design, drives one agent CLI per role episode — so on DeepSeek Harness it can only read the final answer of each dsh --profile headless run, not the intermediate tool events.

This plugin takes the parts that carry the value — independent verification, the downgrade invariant, the verified-state ledger, the mutation guard — and implements them natively in the harness, where the events are available and no POSIX-only primitive is needed. Its workspace-mutation detection is its own contribution; the upstream project guards against verifier writes only through the same prompt-and-allow-list prevention described above.

Verifying this plugin

npm test      # 37 tests: verdict grammar, guard, ledger, tools, extraction
npm run verify   # acceptance probe against the packed artifact

npm run verify is not a unit test run. It rebuilds the package from the files list in package.json, loads the tool set out of that artifact, and runs three scenarios against a real file system with a scripted verifier:

Scenario	What it must show
clean-verdict-is-progress	a clean verdict becomes progress, in one verifier episode
positive-control-mutation-voids-verdict	a verifier that edits a file is caught; the round is blocked / violation and never becomes progress
ledger-survives-a-fresh-context	a brand-new context reads verified progress back off disk

The middle scenario is a positive control: if the fingerprint guard ever stops noticing a mutation, positiveControlTripped turns false and the probe exits non-zero, so a broken detector cannot be reported as a pass. The probe also proves it ran — it records the artifact it loaded, the tool names it found, and that all three scenarios reached a deciding assertion.

The result of the last run is committed: verify/acceptance-report.json and the raw tool results and ledger contents in verify/acceptance-evidence.json.

Requirements and known environment issues

• A subagent provider that does not inherit the parent context. The plugin injects tools and subagents; the spawn provider shipped with the web and headless profiles satisfies this. If only the fork provider is available, the plugin refuses to certify anything and says so in the tool result rather than silently falling back to a verifier that can see the executor's reasoning.

• Windows: a PowerShell that can actually be spawned. DSH resolves pwsh through a fixed candidate list (@deepseek-ai/dsh-pwsh-local); a Microsoft Store PowerShell can leave a zero-byte execution alias in %LOCALAPPDATA%\Microsoft\WindowsApps\pwsh.exe. That alias is a regular file to lstat, so it is selected, and Node then fails with spawn ...\WindowsApps\pwsh.exe ENOENT. If your harness will not start for that reason, point the resolver at a real binary:

  # profile cordis.patch.yml
  - id: pwsh-local
    config:
      pwshPath: 'C:\Program Files\WindowsApps\Microsoft.PowerShell_7.6.6.0_x64__8wekyb3d8bbwe\pwsh.exe'
  

  This is a host resolution issue, not a plugin one, but it is recorded here because it is the first thing that stops a Windows acceptance run.

License

MIT