dsh-self-harness-tools

Self-harness tooling for the DeepSeek Harness ecosystem

_{中文 · English}

Two installable DSH bundles that measure and inspect your own harness:
a probe battery for model priors, and an archive dashboard for release history.

Why

DeepSeek Harness follows an "everything is a plugin" architecture. This repository ships two bundles built on the official Cordis plugin model, installable with dsh plugin add and distributed on npm:

• dsh-prior-probe — frozen probe-battery tooling. It measures how much of an expected answer a model can produce from prior knowledge alone (no-harness, single-shot completions scored with deterministic grader-equivalent regexes).
• dsh-evidence-dashboard — an archive dashboard for your project: components, design decisions, rejected options, and a git-tree of the release history with a changelog timeline.

Features

prior-probe Host plugin with three model tools: prior_probe_list (bundled batteries), prior_probe_score (offline deterministic scoring), and prior_probe_run (no-harness single-shot probing against any OpenAI-compatible endpoint). Batteries are frozen and embedded; api_key_file keeps keys out of session logs.	evidence-dashboard One package with both halves: the host serves the archive over /api/dash-data, the browser half registers a Settings → DSH Evidence section. Tabs: Overview, Decisions, Rejected options, and a Versions git-tree with fork/merge lanes plus a changelog timeline.
Official bundles Each plugin is an npm package declaring dsh.bundle (patch layer + ESM entry); the dashboard additionally declares dsh.client with a pre-built window.__ModuleLoader__ bundle. Install from GitHub, tarball, or npm — no build step required.	Verifiable by design node verify.js — a thin wrapper over dsh-plugin-kit — parses every source, validates every JSON dataset, byte-checks embedded batteries against their originals, spot-checks the grader regexes, and dry-runs both packages, wired into CI on every push.

Quick start

Install both plugins into a profile with the shortest official channel:

dsh plugin --profile demo add dsh-prior-probe
dsh plugin --profile demo add dsh-evidence-dashboard

dsh --profile demo --dump-config   # verify the "# == dsh-prior-probe" layer
dsh web --profile demo             # open Settings → DSH Evidence

Alternatives: from GitHub (dsh plugin add github:SuTang-vain/dsh-self-harness-tools#path:plugins/prior-probe), or from a packed tarball. See each plugin README for details.

Documentation

Goal	Entry point
Install and use the probe-battery tools	plugins/prior-probe/README.md
Install and use the archive dashboard	plugins/evidence-dashboard/README.md
Self-check the repository	node verify.js
Verify, extend, or reuse the QA kit	dsh-plugin-kit/README.md
Release checklist (version bump → npm publish)	CONTRIBUTING.md
What changed in each release	CHANGELOG.md
See the version tree rendered in the dashboard	Versions tab (v1.0 → v1.6.2)

Evidence / 效用证据

Utility data, not promises — what the pieces demonstrably do:

• dsh-plugin-kit: 14/15 checks green across three third-party repositories with three different distribution shapes (npm bundle / official .dsh-plugin / profile snippet); the kit also runs 82 checks in this repository's own CI. See docs/kit-field-test.md.
• dsh-prior-probe: a deterministic offline demo produced by the shipped scoring code ships at plugins/prior-probe/data/example-score-run.json (5/8 scored, G=0.8, low-direction flags visible); live endpoint runs use prior_probe_run with your own credentials.
• dsh-evidence-dashboard: renders this repository's own release history, and the schema is demonstrated on a third-party project's real data — the six npm publishes of @deepseek-ai/dsh in plugins/evidence-dashboard/data/examples/versions-dsh-official.json.

Relationship to the Official Project

This repository is built on deepseek-ai/deepseek-harness and the Cordis plugin model.

The official project provides the core agent capabilities, the plugin system, and the Web UI. This repository primarily provides:

• Two reusable, installable bundles for the official dsh plugin flow
• Tooling to measure model priors before building a task suite
• An archive dashboard with a version tree for project history
• npm distribution with dsh.bundle / dsh.client manifests
• dsh-plugin-kit, a reusable QA kit for DSH plugin repositories

If you prefer to contribute to the Harness core itself, refer to the official repository first.

Community

• Official DeepSeek Harness: GitHub Discussions · Discord
• This repository: report issues and open pull requests on GitHub.

License

This project is licensed under the MIT License.

Community tooling built on DeepSeek Harness. It is not an official DeepSeek product.