dsh-policy

A DeepSeek Harness plugin with two independent jobs:

1. In-stream refusal detection & auto-retry — if the model starts its answer with a refusal, drop that attempt and silently re-issue the request.
2. Policy block injection (optional, off by default) — append a policy text to the outgoing request as a user message at the tail.

The retry piece is the headline feature: it keeps streaming intact, which naive proxies can't do.

中文简介：在 dsh 客户端侧对模型响应做流内判拒并自动重发。只预读开头一两句，被拒那次不会吐给上层，所以正常回复的流式打字几乎不受影响。

Why this is not just a proxy

A reverse proxy sees HTTP text only. To detect a refusal it must buffer the whole answer, wait for completion, then decide — which kills streaming. A client plugin receives structured chunks, so it can decide after the first few text-deltas and discard the rest before the user ever sees them.

plugin:  llm/stream (waterfall)  →  peek first N chars  →  refusal?  →  re-issue & swallow
proxy:   full response buffered  →  judge at the end    →  no streaming at all

Features

• Cheap peek: only the first peekChars (default 200) characters are buffered. Normal answers stream through with a barely noticeable delay.
• Reasoning-aware: reasoning-delta blocks don't count as "output started". A model may think for thousands of characters before refusing — that case is still caught.
• Tool-call aware: if the first block is a tool call, it's not a refusal; no retry.
• Upstream filter aware: content_filter / guardrail_intervened / risk control terminations are treated as terminal. Retrying those just gets cut again and burns tokens, so the plugin stops and annotates the failure instead.
• Bilingual refusal patterns: matches Sorry, I can't…, 抱歉，我不能提供…, 我无法写这类…, and Chinese post-posed negation such as 继母与继子这类家庭亲属关系的性内容我不写.
• Tuned against real DeepSeek-V4.1 output: 20 probe prompts were run against deepseek-v4.1-flash (with a policy block attached) to capture refusals that survive prompt-level persuasion. V4.1 refuses in very short, concrete sentences — 这个我不能写。 / 我不会给你方法。 / 我不能提供甲基苯丙胺的合成步骤… — which generic refusal lists miss because they expect a demonstrative (这类 / 这个) after the verb.
• Guarded against false positives: technical phrasing like 这个函数我不写注释 or 这个类的构造器不能写 const does not trigger a retry.
• Zero dependencies: Node built-ins only.

Install

Drop the plugin directory next to your dsh profile and register it, or install it as a profile bundle (recommended):

# profile/cordis.patch.yml — only needed if you are NOT installing as a bundle
- insert:
    - id: dsh-policy
      name: "dsh-policy"

As a bundle, cordis.patch.yml inside this package already contains that line — don't duplicate it in the profile.

Files are read from ~/.dsh/policy/:

~/.dsh/policy/
├── config.json      # switches (see below)
├── policy-sys.txt   # text injected when mode != off
└── policy.log       # decision log, capped at 256 KiB

Configuration

~/.dsh/policy/config.json

{
  "mode": "off",
  "onlyAgentLoop": true,
  "log": true,
  "retry": {
    "enabled": true,
    "maxRetries": 2,
    "peekChars": 200,
    "peekMs": 1200,
    "minTextChars": 24,
    "textStallMs": 1500,
    "maxWaitMs": 30000,
    "reasoningStallMs": 4000,
    "backoffMs": 350
  }
}

Key	Default	Meaning
mode	off	Policy injection target: off | user | system | both. user is the only position reliably delivered to the model. Retry is independent of this switch.
onlyAgentLoop	true	Inject only on requests that carry tool definitions; leave title-generation and other internal calls alone.
log	true	Append decisions to policy.log.
retry.enabled	false	Master switch for refusal retry.
retry.maxRetries	2	How many times to re-issue after a refusal.
retry.peekChars	200	Characters buffered before deciding.
retry.peekMs	1200	Max wait for the first characters.
retry.minTextChars	24	Below this, keep waiting (a two-character answer is not a signal).
retry.textStallMs	1500	If text stops growing for this long, give up waiting and pass through.
retry.maxWaitMs	30000	How long text may stay empty before giving up — and only if reasoning has also stalled (see next row).
retry.reasoningStallMs	4000	Reasoning must be idle this long, in addition to maxWaitMs expiring, before the peek gives up. A long chain-of-thought that's still streaming means the model is working, not stuck.
retry.backoffMs	350	Delay before re-issuing.

The plugin re-reads config.json when its mtime changes, so switches take effect without a restart. Changing index.js itself does require restarting dsh.

How detection works

looksRefusalHead(text) takes the first 200 characters and tests them against a set of patterns. They fall into four groups:

1. Leading apology + inability — 抱歉，我不能提供…, Sorry, I can't…
2. Leading inability + action — 我不能写…, 我无法生成…
3. Demonstrative + inability — 这个我不能写…, 这类内容我无法生成…
4. Post-posed negation — …这类内容的我不写, 这条线我换掉了

Group 4 exists because Chinese frequently puts the negation after the topic. A rule set that only anchors on 抱歉 / 我不能 at position 0 silently misses those.

Patterns are deliberately narrow: (?=$|[。！？!?，,、；;—…])-style end anchors and a whitelist of object words (这类, 题材, 内容, 关系) keep technical prose out.

Tests

npm test          # runs both suites
node test-retry.mjs        # 16 cases — stream/retry behaviour
node refusal-corpus.mjs    # 47 cases — detection coverage & false positives

test-retry.mjs needs no network and no dsh process — every case drives a fake chunk stream through the real listener:

PASS 判定：开口就拒的形状命中
PASS 判定：正常内容不误判
PASS 第一次被拒 → 丢弃并重发，用户只看到第二次
PASS 先推理很久、正文只蹦出两个字就拒绝 → 仍要抓到
PASS 正文起头就断流 → 不挂死，按断流放行
PASS 一直拒 → 用完次数后把最后一次原样交出
PASS 开头是工具调用 → 不重试
PASS 上游 content_filter 切断 → 不重试，失败信息带注
...
16 项通过

refusal-corpus.mjs is a corpus test built from collected DeepSeek-V4.1 output:

• 22 positive cases — real V4.1 refusals (13 from the probe run + 9 regression cases)
• 5 "complied" cases — same probe run, but the model actually answered; must not be flagged
• 20 negative cases — technical sentences and normal openers that must not be flagged

47/47 通过，失败 0 项

The negative set is the part that keeps the detector honest. Examples it must not flag:

Text	Why
我无法提供具体数字，但可以从三个角度看。	comma → the refusal path is cut off
我无法提供实时数据，但可以按公开资料估算。	same
我不能写死端口，得从配置读。	写 is followed by 死, not a sentence end
抱歉，我无法写得更长。	(?!得) guard
This is beyond the current scope of the project.	only beyond my … / beyond the scope counts
这个函数我不写注释，直接用默认参数。	这个 requires a refusal verb, not a bare 不写

Large-context models (1M)

Long prompts make the model think longer before it writes anything. That broke the original timing rule: maxWaitMs alone expired mid-reasoning, so the peek gave up without ever judging. In one real installation wait-reasoning fired 89 times — 89 chances to miss a refusal entirely.

The rule now needs two conditions: maxWaitMs expired and reasoning idle for reasoningStallMs. A chain-of-thought that keeps streaming keeps the peek open, because the model is working, not stuck.

If most of your traffic runs at 1M context:

{ "retry": { "maxWaitMs": 45000, "reasoningStallMs": 4000 } }

Cost note: a retry re-sends the entire prompt. At 1M context that's another ~1M input tokens per retry, so keep maxRetries at 1–2 if you pay per token. The plugin does not deduplicate prompt caching — it relies on the upstream's own prompt cache.

Extending the corpus

tools/probe-refusals.py runs 20 refusal-inducing probes against any OpenAI-compatible endpoint and dumps the answer openings:

export WB2API_BASE=http://127.0.0.1:7863
export WB2API_KEY=sk-...
python3 tools/probe-refusals.py --model cn:deepseek-v4.1-flash

Read refusal-samples.json, find openings the matcher misses, add them to REFUSAL_OPEN, and record them in refusal-corpus.mjs so they never regress.

Two practical notes:

• Reasoning models: keep --max-tokens at 1000+. Too small and the reasoning pass eats the whole budget, producing empty content that looks like a refusal but is just truncation.
• Aim at the assistant's own wording, not the topic. The value of the probe is the sentence shape (我不会给你方法), not which subjects get refused.

Limitations

• Retry is a statistical fix. If the model refuses every time for a given input, retrying twice just costs tokens; the last attempt is returned unchanged.
• Detection is prefix-based. A refusal that appears after several paragraphs of normal text is not caught.
• The peek window adds latency proportional to peekMs on slow first tokens — lower it if you care more about time-to-first-character than detection accuracy.

License

MIT