DSH HUB
HomePlugin StorePlugin PacksCommunityRankingsResourcesPublish Guide
Plugin source
Back to catalog

PhilYue /

PhilYue/dsh-llm-manage

Verified

DeepSeek Harness: ds-llm-manage. 模型管理(收藏、置顶、探活),熟练度排序,故障自动切换,防止长任务中途中断

★ 0 Stars0 Forks0 IssuesN/A Community rating0 Confirmed installs
View on GitHub
READMESource: master@9e39a982

dsh-llm-manage

A persistent DeepSeek Harness plugin that manages the model inventory of a configured LLM route: pins and favorites, a per-model reason why each model is unusable, gateway directory reconciliation, and automatic fallback when one model's quota runs out.

No screenshots are committed, on purpose. screenshot.mjs captures a running harness, so the images show whatever routes and model inventory that harness has — the maintainer's own deployment. A route name rendered into a PNG cannot be found by a text scan, which is how a set of them was published once. Run the script against your own harness (see Looking at the panel without restarting) and you get your own routes, which is the only version of that picture that means anything to you.

Why

A route that aggregates many providers behind one gateway exposes its whole catalogue as one flat list. Five things then go wrong at once, and none of them is the gateway's fault:

Pain What this plugin does
A long list, favorites buried Pin and favorite models; the picker lifts them into their own sections at the top, and the panel sorts pinned → favorite → recent → rest
No sense of which models you actually rely on A local ledger levels each model by how often you use it and ranks the rest by 14-day heat, shown as a Most used card in the panel and a ranked section in the picker
Retired models still listed; selecting one errors Reconciles the route's models: list against the gateway's own GET {baseURL}/models directory and flags anything missing as retired
Quota exhausted, manual switching Watches agent/request-error, moves the session to the next usable model on a model-scoped failure, and reports the switch only once the model that answered confirms it
The same model served by several providers Marks each row with how many other routes also serve it, instead of hiding the copies
Models that cannot be called at all Probes with a minimal request and labels the incompatibility honestly

Nothing here is hard-coded to one gateway or one model id.

Install

From the npm registry (the packaged release, no toolchain needed):

dsh plugin --profile web add dsh-llm-manage

dsh plugin forwards the arguments after --profile <name> to pnpm inside that profile's directory, so anything pnpm accepts works here (add, remove, why, …). The profile maps node_modules/dsh-llm-manage to the package from the registry.

Development installs

Prefer the npm package for a normal installation. Two alternatives are useful while working on the plugin itself:

  • Install from a path to link the directory, so local edits show up without reinstalling:
    dsh plugin --profile web add /absolute/path/to/dsh-llm-manage
    
    This symlinks the working copy — ideal when you are the author. Every edit is live, which is why "Applying a change" below is written for it.
  • Install from the git URL to get a fixed copy of the latest source rather than the npm release:
    dsh plugin --profile web add git+https://github.com/PhilYue/dsh-llm-manage.git
    

The package installs as a bundle: its dsh.bundle.patch row inserts one Host plugin, and its dsh.client declaration makes the web shell load the browser half. The Host half needs a full harness restart; the browser half is picked up by the client HMR watcher, so a page refresh is enough. See "Applying a change" below.

Configuration

Every value lives in the plugin row's config, so your own patch layer keeps overrides across upgrades. Defaults:

- id: llm-manage
  name: 'dsh-llm-manage'
  config:
    selector:
      mode: managed     # our picker owns the seat; 'native' hands it back to the shipped one
    probe:
      intervalMs: 1200  # delay between deep probes
      timeoutMs: 30000
      batch: 40         # probes accepted per request
    directory:
      ttlMs: 1800000    # how long a gateway directory read is trusted
    fallback:
      enabled: true
      chain: []         # 'route/model' entries, tried first
    recovery:
      retryFirst: true  # let the platform retry a transient failure before we move
      autoContinue: true  # pick the work back up once the model has moved
      maxSwitches: 3    # automatic switches allowed per request (0 = never move)
      maxContinuations: 2  # pickups allowed per request (0 = move and stop)
      idleWaitMs: 1000  # how long to wait for the session to go idle before giving up
    usage:
      enabled: true     # local proficiency/heat counters
      seed: true        # one pass over existing session logs at startup
      flushMs: 5000     # counters are batched to disk on this interval

The fallback chain order is: fallback.chain, then pinned models, then favorites, then the remaining models on the same route, then other routes. Models already known to be retired, protocol-incompatible, hidden, or inside their quota cooldown are skipped. At most three automatic switches happen per turn by default — see "Budgets are policy, not constants" below for the three numbers and where to change them.

The host has to be new enough

package.json declares engines.dsh. DSH reads it before installing and before activating a plugin, and it decides compatibility on its own (peer dependencies are ignored once it is present), so a host outside the range fails at install time instead of installing and then quietly rendering less than it should.

That matters more here than it usually does: this plugin takes over UI seats (conversation.input.model and the session options that hang off it), a projection (agentTeam), and SDK calls (agent.followup, sessionProjections, remote.agentTeams) that only exist from 0.2.2 onwards. Installed into an older host it would still load, still look right, and silently drop the rows whose plumbing is missing — which reads as "the plugin forgot to implement it". A refused install is the honest failure.

Budgets are policy, not constants

Three numbers used to be constants in the Host. They are the ones an outage actually makes you want to change, so they are now config, and editable from Model manager → Failover → Automatic handling:

Setting Default What it bounds
switches per request 3 automatic model switches inside one request
continuations per request 2 pickups of the interrupted work (0 = move and stop)
idle wait (ms) 1000 how long we wait for the session to go idle before giving up

0 is a real value, not "unset": zero switches means never move automatically, zero continuations means move but leave the work for you. Writing any of them stores an override in the plugin's own config document — never in your cordis.patch.yml — and the dialog reads the effective value back from the Host, so what you see is what the next failure will use. Out-of-range input is refused both in the field and at the Host (400), because a number that silently becomes something else is worse than a rejected edit.

Which failures switch, and which never do

The rule is scope, not severity:

Failure Scope Switches? Recorded against the model?
model not found / retired one model yes yes
wrong protocol for this route one model yes yes
per-model quota exhausted one model yes yes
needs authorization on this account (401/403, OAuth, star-gated) one model never yes
other per-model failures one model yes yes
no adapter registered for this route the whole route yes no
malformed request / context too long the request itself never no
account-wide rate limit the whole API key never no
account-wide quota the whole API key never no
the route does not answer at all (DNS, TLS, refused socket) the whole route never no

A model-scoped failure moves to another model, because a different model can serve the turn. An account-scoped failure never moves: every model shares that key, so switching would only hide the real problem and burn another request.

A switch cannot rescue the turn that just failed. DSH freezes the provider, model and reasoning effort at step preparation, re-validates them when the request is built, and replays the same frozen selection on a retry — so the plugin commits the switch with agent.model.select(), the one path the SDK documents, and the work continues on the new model (see below). The failed turn reports its real error instead of a request-time mismatch. If the Session still holds the broken selection when the next turn is claimed — a refused controller call, or a level the session did not drop — the repair is finished there, synchronously, before the step reads its selection.

A route with no adapter is its own scope. Switching models on that route cannot help — every model there fails the same way — but switching to another provider can, so Two of those scopes were learned the hard way, from a real log:

  • Needs authorization (AUTH) is a fact about this account and this model: the gateway answered, and the answer was "not for you" — an OAuth grant nobody completed, a model behind "star the repo first". Switching models cannot fix it, so it never switches, and burning the continuation budget on it would only produce the same answer twice. It is still recorded against the model, because "this one needs authorization" is exactly what the next fallback chain should skip.
  • The route does not answer (NETWORK) is a fact about nothing in particular: DNS never resolved, the socket was refused, the certificate did not verify. The model was never asked, so blaming it would remove a perfectly good model from future chains.

it still switches. What it must not do is land in the model's health record: error is in the set of reasons that make the fallback chain skip a model, and unlike quota that verdict has no staleness escape, so a single route misconfiguration would have removed an innocent model from every future chain permanently.

A probe is not a routing event

"Unusable" and "move the conversation" are two different conclusions, and only a failed request reaches the second one:

  • A failed request (agent/request-error) is the live conversation hitting a wall. If the failure is model-scoped, the turn is retried on the next usable model and a banner names the model it moved for.
  • A failed probe is an answer to a question you asked about a model. It records that model's health — so the next real switch skips it — but it does not touch whichever session happens to be using it. Moving a live conversation because of a diagnostic you ran would be switching models behind your back, which is the thing this design forbids.

Concretely, with model-gamma selected and its quota exhausted:

Event Selected model afterwards
you deep-probe model-gamma, and the probe fails still model-gamma
a real request on model-gamma fails with per-model quota model-beta (next usable), banner shown
a real request fails with an account-wide key limit (2008) still model-gamma

At most three automatic switches happen per turn, and if the chain finds nothing usable the failure is passed through untouched rather than retried in a way that cannot succeed.

Picking the work back up

Switching the model is only half the job: the turn that failed is gone, and until now this plugin stopped there and told you to send the next message yourself. That is the wrong place for the work — you asked the question to get an answer, not to learn which model answered it.

So when a switch is committed for a failure that interrupted work, the plugin queues the next turn itself, with agent.followup() — the SDK's own path for "make this the sole ordinary message of its own turn". The queued message does not pretend to be you: its source is llm-manage-repair, and it says what happened and what to do about it. It asks the new model to check the actual state first — the files it wrote, the task list it updated, the command output it was reading — and only then continue, without repeating work that already landed. A turn that died mid-tool-call may have finished its side effects; continuing as if it had not is how a recovery turns into a mess.

What that buys, and what it does not:

  • The queue is only filled at the turn boundary (turn/end), after the switch is a fact.
  • If a message is already waiting — you typed while it was recovering — the plugin does not add another; yours gets answered on the new model, which is what you wanted anyway.
  • At most two continuations per request. A model that fails, is replaced, and fails again is a fact about the route, not about the turn; three identical attempts is not resilience, it is a loop. Typing a new message resets the budget, because that is a new request.
  • If it declines to continue it says so, in the same place it used to say "next message only" — the banner names the reason instead of leaving you to guess.
  • "Took over" is a claim with evidence, not a hope. The turn boundary can only prove that the frozen turn did not come back, so the record is written as did not take over and keeps its id. When a later turn finally produces a message on the target model, that same record is amended to took over, with the time it happened. Records written before this version have no id and are marked legacy in the history rather than silently counted as failures.
  • Ordering is deliberate: an ordinary transient failure goes to the harness's own retry policy first (it is bounded, it knows the backoff), and only a failure that survives it is treated as a model problem worth moving. A request that is malformed, too large for the context, or wrong in shape is never a model problem and never switches — that just spends another model's quota on the same invalid request.

Both halves are exposed where the rest of the policy lives: Model manager → Failover → Automatic handling, two switches with their trade-off written underneath. They write the moment they move (a switch that only took effect after pressing a Save button somewhere else would be lying about what the plugin is doing), and they are stored in the plugin's own config document — the same one the fallback chain uses, so cordis.patch.yml is never edited behind your back. The Loader row still works as the default: recovery.retryFirst and recovery.autoContinue, both on, and the UI switch simply overrides it.

The same dialog now also carries the numbers behind that policy — switches per request (default 3), continuations per request (default 2), and how long to wait for the session to go idle before giving up (default 1000 ms). They are ordinary number fields: Enter or blur commits, an out-of-range value is refused on the spot instead of being sent, and 0 is a real value ("switch nothing", "continue nothing"), not an empty box.

One more check belongs here: if the default model — the one new sessions start on — is itself retired, missing from the gateway directory, or sitting on an unhealthy record, the picker says so in one line and offers the model the fallback chain would pick, with a button that writes it. Otherwise every new session starts on a model that is about to fail, and the recovery machinery is doing nothing but cleaning up after a choice made somewhere else.

The line states the reason rather than a verdict — quota exhausted, needs authorization, route unreachable — because "cannot be called" sends you looking for a network problem when the account has simply run out of free usage; the raw provider text stays in the tooltip. It also says whose problem it is: a conversation that never picked its own model inherits the default and is told so, while one that chose another model gets "this conversation keeps its own model" instead of a warning about somebody else's default.

The switch happens, and the banner proves it

Two mechanisms move a model, and the difference matters:

  • The session's own selection (sessionController.selectModel) is what makes the change real. It validates the target against a registered adapter and appends a durable model/selection event, so the model the conversation uses actually changes and every surface agrees. The saved default model is put back afterwards: an automatic recovery has no business permanently changing which model new sessions start on.
  • A per-request override, bound to the exact turn and step it was decided for, is what recovers the turn already in flight. It is needed because the harness applies the session selection during prompt assembly, which happens once per step — before the retry loop — so the session change alone would only take effect on the next turn.

The banner then reports what was proven, not what was decided. A switch is recorded as pending, and only promoted to something you are shown once a committed assistant message names the model that actually produced the response:

What the answering model was What the banner says
the model we switched to, work queued again Model switched and the work was picked back up
the model we switched to, nothing queued Model switched, with Switch back
a different model Switch did not take effect, naming the model that actually ran
no message — the turn ended Switch attempted, turn not recovered

The earlier version wrote the banner when the switch was decided, so a retry that failed, was aborted, or ran on a third route produced the same "switched automatically" text. It also hardcoded "exhausted quota" as the cause for every failure, including failures that were not about quota at all. Each reason now has its own wording, and the provider's raw message is one clamped line with the full text on hover — inlined in full it was up to 400 characters, which made the banner taller than the conversation around it.

The banner carries the way forward, not just the news. When the plugin could not pick the work back up itself — automatic recovery is off, the budget is spent, or a message was already waiting — Continue appears instead: it fills an editable prompt ("continue from where the previous answer stopped") into the draft and persists it, so the decision to resend stays yours. It never sends on your behalf. Switch back is offered under the same condition, since it moves the session's model back. Stop switching suppresses automatic switching for that session until the harness restarts. Dismiss hides the report for that switch only.

Some gateways report a model's quota exhaustion as HTTP 429, which the harness's own classifier maps to a retryable RATE_LIMIT — indistinguishable from an account-wide rate limit. The two need opposite handling, so this plugin reads the gateway's numeric business code out of the error body instead of trusting the mapped code. The code table is at the top of index.js; adjust it there if your gateway numbers things differently.

Why probing is manual

GET /v1/models is free but is only a directory, not a callability list: a model can be advertised and still fail on first use (wrong protocol, no capacity, image-only). Probing each model deeply is therefore the only way to learn whether it works — and deep probes spend the gateway's per-key rate budget, so a sweep large enough to be useful is also large enough to rate-limit the user out of their own gateway.

Health is learned cheapest-first instead:

  1. the free directory set,
  2. real request failures as they happen (passive, zero extra traffic),
  3. explicit probes the user starts, throttled at probe.intervalMs and capped at probe.batch.

Each row's status is the strongest evidence available, in this order:

Status Means Shown when
retired the gateway no longer lists the model the directory read succeeded and the model is absent from it
the recorded health a probe or a real request produced a verdict any health record exists for that route+model
ok the gateway lists it, and no failure is known the directory read succeeded and there is no health record
unknown nothing is known the route exposes no baseURL, so there is no directory to compare against

ok therefore means "listed and not known to be broken", which is weaker than "verified working" — a model can be advertised and still fail on first use. The panel keeps the distinction visible by showing the source alongside the status, so a row reads usable · deep probe once it has actually answered and usable · never checked while it has not. Neither is hidden: the second is the honest label for the majority of a large catalogue, because verifying all of it at once is what would rate-limit you out.

Verifying it

Five suites, all runnable without any particular deployment:

node verify.mjs          # Host half: registry, gateway, classification, fallback
node verify-client.mjs   # Browser half: renders the panel in jsdom and clicks it
node verify-browser.mjs  # Control geometry in a real engine (headless Chrome)
node verify-parity.mjs   # Did the shipped seats we take keep their capabilities?
node verify-live.mjs     # The whole thing against a RUNNING harness (opt-in)

verify-parity.mjs needs the harness source (DSH_CHECKOUT) and fails without it, because its whole job is comparing this plugin against the shipped one. See docs/upstream-parity.md.

Each suite picks a mode automatically:

  • real — this machine has a profile whose first provider route resolves, plus that route's credential. The settings under test are then the deployed ones, and the gateway directory is read live. This is the mode that catches a fixture agreeing with a wrong assumption.
  • synthetic — otherwise. A local stub serves a synthetic catalogue, so every code path still runs: offline, deterministically, with no credential and no internal endpoint.
LLMM_VERIFY_MODE=synthetic node verify.mjs    # force a mode
DSH_CHECKOUT=/path/to/deepseek-harness node verify.mjs

DSH_CHECKOUT points at the harness checkout, which supplies the real Tool registry, React and jsdom. It is not vendored here; the suites try a few conventional locations and then tell you what to set.

verify.mjs also gates the publish surface: where the maintainer's company-information scanner is present it fails if the runtime files (index.js, client.js, package.json, cordis.patch.yml, icon.svg) contain anything identifying a particular employer or internal service; in this published package the scanner is absent, and the gate says so rather than reporting a pass it did not perform. The scanner is not shipped because its own rule patterns have to quote the identifiers they look for.

Some checks exist because the live harness caught bugs a shallower suite passed:

  • The suites execute the registered tool, not just register it. A definition can register cleanly and throw on its first call.

  • The fixture context enforces Cordis's inject guard, so reading ctx.llm from apply's own context fails exactly as it does live.

  • Never test one half alone. The two halves can agree on every type and still disagree on a field name: the panel once folded data.snapshot while the Host returned the same state under value, so pin/favorite/hide updated nothing until the user reloaded by hand. A Host-only suite passed it, and so did a client suite whose other half was a mock returning whatever field the client asked for.

  • verify-browser.mjs exists because jsdom has no layout: every getBoundingClientRect() returns zeros, so a tooltip's position and a button's width cannot be measured in-process. It loads the real panel into headless Chrome and asserts the geometry the design depends on, then skips with exit 0 when no Chrome-family binary is present (CHROME_PATH overrides).

  • verify-live.mjs exists because every other suite builds its own page and hands the plugin a context it controls — so none of them can catch a contract they do not enforce. The plugin declared slots, modelDirectories and sessions at the top level, then called modelDirectories.directoryFor(), which internally reaches this.ctx.remote.session. Cordis checks the consumer's inject list, so in the live app that read threw cannot get property "remote.session" without inject, the picker's slot entry crashed, and because Style renders inside that same subtree the composer's model control lost every rule in its stylesheet. All three suites were green: the hand-built contexts were plain objects that enforced nothing.

    It opens the URL dsh web prints — in a real engine, against the real profile, real catalogue and real composer — and skips with exit 0 when no live harness answers:

    DSH_LIVE_URL='http://127.0.0.1:3080/?token=...' node verify-live.mjs
    DSH_LIVE_URL=http://127.0.0.1:3080/ DSH_LIVE_COOKIE='dsh-auth-...=...' node verify-live.mjs
    

Two more contract mistakes the suites had to learn

Both of these shipped, both were green in every suite, and both were found by looking at the running app. They are worth stating because each one changed what the suites assert.

The theme cannot be supplied by the thing being tested. The card was painted with --dsw-alias-bg-overlay, which the plugin token contract labels "Overlay and popover background" — a reasonable reading of the docs. Measured in the app it is #e9ecf2, a mid grey, darker than the white composer it drops out of; DSH's own popovers paint the translucent --dsw-specific-menu on a child layer with a backdrop blur instead. The suites could not catch it because they defined that token themselves: verify-browser.mjs had --dsw-alias-bg-overlay:#fff, and the deleted preview.mjs invented the same white. The suites now carry the values the app actually resolves, and verify-browser.mjs fails if the stylesheet reads any token its theme block does not define — 27 are read. verify-live.mjs asserts the painted surface against the token's real value.

A stylesheet in a template literal can truncate without a syntax error. A backtick or ${ inside a CSS comment ends the literal early. The pieces often still re-parse as JavaScript, so node --check passes, and the failure that reaches you is a browser refusing the import — the whole client half silently absent. verify-client.mjs extracts the literal and asserts it is intact, which catches the case that parses but is wrong.

The lesson generalizes past this bug: a suite that supplies the context cannot verify the context. An inject declaration is a contract with the runtime, and only the runtime can check it.

Looking at the panel without restarting

DSH_LIVE_URL='http://127.0.0.1:3080/?token=…' node screenshot.mjs

Captures the panel and the model picker out of the running harness, light and dark, into docs/ — which is gitignored, so the images stay on your machine. It is the preview of record because it supplies nothing: it opens the URL dsh web printed and photographs what is there.

The images are not committed because a capture of a real harness is a picture of the routes and model inventory that harness is configured with. That is the maintainer's deployment, and it is also the one kind of content a text scan cannot inspect: a route name rendered into a PNG reads as PNG bytes, matches no rule, and passes. Four such screenshots were published before anyone noticed. build-public.mjs now refuses to stage any binary asset, and scan-company-info.mjs reports files it cannot read as unscannable rather than counting them as clean — the whitelist that decided which extensions to open was the whole defect.

This replaced a preview.mjs that rendered the plugin into a hand-written page with a hand-written theme. That page defined --dsw-alias-bg-overlay:#fff — a value the token never has in the app, where it is #e9ecf2 — and defined only the 14 tokens the plugin contract documents while the stylesheet reads 27. The mock showed a white card; the app painted a grey one. A preview that supplies its own theme cannot show you the theme, and an approved mock is worse than no mock: it is the reason the grey card shipped.

The replacement supplies nothing of its own and skips with exit 0 when no harness answers, so it never reports a picture it did not take. To reach the panel it sends the settings.open chord — primary+alt+Comma on web, not the plain primary+Comma the desktop runtime uses — then clicks through the settings nav until the panel appears, matching no label because the labels are localised. A development aid, not a test.

Where it appears

  • Settings → Models → the provider card: the management panel.
  • The composer model picker: this plugin's picker in the shipped picker's seat — one entry, one panel, with the shipped capabilities (search, provider grouping, keyboard roaming, a trigger that names the current model and level) plus levels, probes, the manager and the usage figures. selector.mode: native hands the seat back to the shipped picker and leaves only the health dot, level chip and listing badge on its rows.
  • Above the composer: a banner when this session's model was switched.

Decorating the shipped picker

As of DSH 0.2.2 the shipped picker carries its own reasoning levels, its own badges and the provider-default effort, so this plugin stopped shadowing it: the seat belongs to the owner and we contribute the marks the owner cannot know — the health verdict, the proficiency level use has earned, and the listing badge.

That is a slot contribution (conversation.input.model.label), and two of its rules decided the implementation:

  • A contribution replaces the owner's fallback, so the model name is ours to draw. A label that returned nothing would blank the composer control — the failure is invisible in a picker screenshot, so verify-client.mjs asserts the name is rendered.
  • The label mounts once per catalogue row, so the plugin's stylesheet is mounted once into <head> (re-found by attribute) rather than per instance: a sheet per row would multiply 20 KB by the size of the catalogue. A second mount is asserted to reuse it.

The level only appears once use has earned it (level 2 and up) and only where there is room: the collapsed trigger keeps the health dot alone, because the owner budgeted its width.

The composer controller

Inside the composer card, on the row the submit button lives on, sits a control cluster that changes shape with the session instead of holding a fixed row of buttons:

  • idle — the manager button and a dimmed ⌘K; nothing else. The reasoning level is deliberately absent: the model trigger a few pixels away already names it and opens the panel that owns it, and one fact with two controls reads like two facts.
  • typing — adds Draft 42 chars, plus attachment and queue counts when they exist.
  • a submit in flight — the phase it is in: reading command, command claimed, running command, sending.
  • a running turn — Running 1:07 with a status dot.
  • waiting on you — the same line, followed by waiting for you.

It is deliberately additive. conversation.composer.bar owns the editor — IME composition, paste, slash adjudication, @-mentions, attachment drops, local echo, the hero variant — and replacing it would mean reimplementing all of that while losing every shipped conversation.input.* seat that renders inside it. The cluster lives in conversation.input.right — the list the shipped editor already reads for "compact controls before the submit action" — so it sits in the card instead of floating above it, and reads the same live input state the editor publishes: it reacts without owning a keystroke.

Taking the model seat means inheriting its obligations, not only its space: the shipped menu rendered conversation.input.model.options there, which is where the agent-team toggle registers. The picker renders that slot with the same props (locked, dismiss, reportFailure), so switching a team on still happens where it always did — a capability that vanished because a replacement did not know the slot existed would be worse than not replacing the menu at all.

⌘K (or ⌘J) opens the command palette: a session console, not a chat box. Its rows are grouped — This session, Model, Diagnostics, Manage — because a flat list of eleven rows with eight of them being reasoning stops is a list nobody reads.

  • Search filters as you type. Matching is by subsequence over label, hint and alias, so th finds 探活 and mdl finds the dialog row; every row carries English and Chinese aliases, which makes typing one word faster than reading a list. A word that matches nothing says so instead of drawing an empty card.
  • Levels are one row naming the stop actually in force (Level · High), and that row opens a second page: the stops plus Auto, with exactly one of them marked as current. ‹ or Esc goes back. The stops used to sit in the root, where six of them outnumbered every other command and pushed the useful rows below the fold.
  • A command in flight says so. The running row carries a spinner, every other row is disabled, and the answer — or the failure — lands on the row that was asked. Previously the palette tracked a busy flag in state and then never read it while rendering, so a probe that took a round trip looked like a dead row you could click again.
  • Diagnostics got the rows only it can offer: refresh the gateway catalogue (a snapshot that bypasses the Host's cached directory), probe every model on this route (batch probe, capped at 40 upstream, submitted and then reported in the probe log rather than blocking the panel for a minute), plus Recent failovers and Probe log, which open the manager on that tab.
  • Escape peels one layer at a time: clear the search word, then leave a sub-page, then close.

Rows are the only things in the palette that are not the search box, and a row that names a tab lands on that tab — opening the dialog on the catalogue and making the reader click again answers a different question than the row promised. Entry points that name no tab (the ⚙ button, ⌘⇧M, the picker footer) land on the catalogue even if the last visit sat elsewhere.

Every row runs a real action through the seats the owner already provides (inputActions.submit, inputActions.setDraft, directory.select, the plugin's own Host route), so the palette never renders a command it cannot execute. Enter runs the highlighted row and stops the event there, so it cannot also submit the draft behind it. The palette is a fixed surface: the composer card clips its own children, so a popover born inside it would be cut at the card's edge.

The palette is drawn from the same material as the model picker card — the theme's menu fill on a ::before layer behind the content, the elevation stroke, border-l1 for the rule under the header. An earlier version filled it with --dsw-alias-bg-overlay, which is the scrim colour behind a dialog rather than a surface colour, so a 560px panel painted with it read as a different product pasted over this one.

The composer model picker

The model picker, rendered from a synthetic catalogue

The picture is a synthetic render, not a photograph of anyone's harness: shots.mjs puts the real client.js in front of a real engine with the app's own theme tokens and an invented catalogue — fake models on fake routes — so a published screenshot cannot contain a route you do not already have. That is also why shots.mjs exists next to screenshot.mjs: the latter drives a running harness, which is the right tool for seeing your own catalogue in your own theme, and the wrong one for a public image, because it photographs whatever routes and models the author happens to have configured. Regenerate the images here with DSH_CHECKOUT=… node shots.mjs.

The panel, from the same renderer:

The models panel, rendered from the same synthetic catalogue

The picker opens on Pinned, then Favorites, then Most used, then one section per route. Pins and favorites are lifted out of their route and shown once, at the top, so a favorite is one click away no matter how many routes the profile has or how many models sit ahead of it alphabetically. Below the sections, each route still lists everything else, minus the rows already lifted — every listed model is offered exactly once. Hidden models are not selectable, but they are not discarded either: they collect in a Hidden section at the bottom, which is the only place the menu can offer them back.

A model that is both pinned and favorited appears under Pinned only, matching the fallback chain's own precedence, where a pin outranks a favorite.

Most used is earned rather than declared: it holds the five models this profile reaches for most, ranked by heat, so the models you actually work with rise to the top without being pinned by hand. It is capped, because it is a shortcut to the top of the list rather than a second copy of the catalogue, and it never repeats a pin or a favorite.

Because a lifted row is no longer under its route's heading, a section that spans more than one route prints the route beside each model. The same model id on two routes is two different selections, and the row has to say which one it means. A section that stays on one route prints nothing, since the label would be noise on every row.

Rows keep their health dot and reason chip, so an unusable model is visibly unusable here too — but it is still selectable. The plugin's view of a model can be stale, and hiding a model from the picker is the one thing this design refuses to do silently. Search filters across the lifted sections as well as the route groups.

A model that has been used carries a proficiency chip beside its name (Lv.3) and, when this week differs from last, a trend arrow with the 7-day count attached (↑12, ↓4). The count is the part that carries the meaning: an arrow alone cannot be told apart from one extra request. A model that has been used carries no trend at all once it has been idle for a week — there is no recent use to report, and a flat arrow would invent one. A model that has never been used carries no chip at all: a wall of Lv.1 badges on an untouched catalogue would be noise rather than information, so the chip itself is the signal that there is history behind a row.

Most used carries a distance-to-next-level bar under each row, because it is the one section built entirely from usage and the one where the models you are actually choosing between live. The bar is a hairline on the row's bottom edge, so it costs the list no height. Each row's tooltip spells the whole figure out — level, uses, the count that reaches the next level, tokens in/out, and whether the number came from the history scan.

Once any model has usage, a Sort control at the foot of the card orders the rows inside every section: the default designed order, By proficiency, or By recent use. It sits outside the scrolling list so it stays reachable, and it exists only when there is usage to sort by — on a fresh profile it would offer three orderings that all look alike. The two usage orderings answer different questions, and on a real profile they disagree: the model you have used most is often not the one you are reaching for today.

The session options the shipped menu carried

The shipped menu renders two child seats under its root: the model label, and a list of session options. The list has exactly one occupant today — the agent-team switch (「启用智能体团队协作」 / Enable agent teamwork) from ui-agent-team.

Taking the seat takes those children with it, and a replacement cannot re-declare them: a child seat has one declaration and the shipped entry owns it. The options list therefore has to be re-provided by hand, against the same projection and the same remote call the shipped switch uses — which is what the row at the foot of the picker does. It keeps the shipped copy verbatim (compared against upstream every run, both languages), hides itself for a child session and when the capability is absent, and asks before turning teamwork off once the session has already run a model choice, because the next turn only keeps its cache if it keeps the same tools. The confirmation is inline in the menu rather than in the shipped modal store, which this plugin does not own.

If a future harness lets a replacing entry inherit children, that row should be deleted and renderSlot used instead. Until then, docs/upstream-parity.md records the rule and verify-parity.mjs enforces it: every shipped occupant of a child seat we inherit must be accounted for, and an unlisted one fails the suite — which is how a capability stops disappearing quietly.

Managing a model without leaving the conversation

The picker is where you look at models most often, so the moment you notice one is unusable is the moment you want to act on it. Every row therefore carries a chevron that opens its management block: pin, favorite, hide, and deep probe, each with a label rather than a bare icon, above the same record the panel's detail block shows — the two surfaces render it from one component, so they cannot describe a model differently.

The chevron is the whole cost of this on a row. Four always-on icon buttons would take about a hundred pixels from a menu that is between 240 and 420 wide, and the model name is the most important thing in it; they would also turn one click target into five, so a near miss on "select" would silently hide a model. A chevron is one glyph, and it keeps every action reachable by pointer, keyboard and touch — where a hover-only strip would be reachable by pointer only.

Two consequences of putting management inside the picker are handled explicitly:

  • Pinning does not move the row between sections. Freezing matters more here than in the panel, because a pin lifts a row clean out of its route group into Pinned. Section membership is frozen at the first pin or favorite and settles when the menu is next opened; what the row reports stays live, so the button lights up immediately. Freezing the row's own flags — the first thing this did — also froze the pin indicator, and the menu claimed the click had done nothing.
  • Hiding is not a one-way door. A hidden model leaves the menu, as hiding means it should, but the menu collects hidden rows into a Hidden section at the bottom and offers Unhide in the same block. Filtering hidden rows out of the catalogue entirely — the original behaviour — made hiding from the picker permanent unless you went to settings.

A hidden model still lists in the panel, dimmed, where the same action brings it back.

The picker is styled from the panel's own vocabulary rather than as a separate design: the same row box (padding, radius, hover background and hover border), the same dimming for a de-emphasised row, the same focus ring, the same .llmm-chip for the route as the panel uses for "also on", and section headings in the secondary label color. The brand color stays reserved for state — the panel's active controls and the picker's current row. Two suites assert this property by property, so the two surfaces cannot drift apart again.

Reasoning levels

A model that declares reasoning efforts gets a level segment on the trigger, beside the model name: · High ▾. Clicking it opens the levels that model declared, and the choice applies to the current session — no second menu, no settings round-trip.

Two different facts decide what that segment can offer, and conflating them is the trap:

  • Measured — what the gateway accepted when the model was probed. A probe sweeps minimal through max, one tiny request per level, and keeps both the accepted and the refused answers.
  • Declared — what the provider's reasoningEfforts config names. The harness validates a switch against this, so a level the gateway accepts but nobody declared is offered by nobody: the segment reads default and cannot be changed, however good the probe's answer.

A probe closes that gap by itself: when the sweep measures any level, the answer is written back through the Host's settings service in the same call, so the levels the gateway accepts become switchable without a second click. A route whose settings the Host cannot edit simply gets no declaration — the probe's verdict still returns, and the detail block keeps its manual Use measured levels action as the fallback.

The write merges, and since DSH 0.2.2 that is not a stylistic choice: the pi-ai schema reads a declaration as closed, so every level the dict does not name is pinned to unsupported. Replacing the dict would therefore delete facts the probe never measured — the official off level, a wire spelling that is not the level id, rows another editor wrote. Only missing levels are added, off is never invented (an unset level already means "send nothing", which is what this plugin measures as the gateway's ceiling, so declaring off would be a claim it cannot support), and a model the platform already offers every measured level for is left untouched entirely: no write, and already: true in the answer.

The rollback is unchanged and narrow: delete that model's reasoningEfforts keys.

A level the provider refuses is repaired rather than repeated — but the repair is to clear the level, not to pick another: DSH 0.2.2 gives "no explicit level" its own meaning (the adapter default, validated against the declaration), so re-pinning one would turn an automatic setting into an explicit one. The session is moved once per turn and the retry carries no level.

The session row underneath

The shipped menu ends with a row of session options, and the one that exists today is Enable agent teamwork. We cannot inherit it: the renderer only hands an entry the renderSlot prop when that entry declares the child slots itself, and the slot table refuses a second declaration of a key the shipped entry already declared. A replacement therefore renders fewer things, silently.

So the row is drawn here instead (TeamModeRow), driving the same projection and the same remote method the shipped one does, and matching its behaviour point for point: hidden for a child session, hidden while the projection is missing, and a confirmation before switching it off once a model has been chosen in this session — turning teamwork off mid-conversation stops the shared task tools, and that is worth one question. The copy is the shipped copy, verbatim, in both languages.

This is a debt, not a design preference: if the framework ever lets a replacement inherit a seat's children, delete the row and call renderSlot again. The parity check below exists to make sure the opposite case — a row appearing upstream that nobody reproduces — fails loudly instead of going quiet.

The model manager

A dialog for the questions a picker cannot hold: which models may replace a failing route, what proficiency each model has earned, and what the last probes said. It opens from the picker's footer, from Ctrl/⌘+Shift+M anywhere in the session, and from the composer palette's Manage models row — the last two matter because the shipped picker is the default, and a dialog reachable only from our own picker would be unreachable.

Four tabs, one draft:

  • Catalog — the same model-first list as the picker, with the four tools inline (pin, favorite, visibility, probe) and a row that expands into the proficiency editor beside the reasoning source and the connection summary.
  • Failover — the ordered candidates, the enable switch, and a local "check the chain" that answers from the catalog and the recorded health instead of spending a request per candidate. Four notes above the button say what the policy will and will not do. Adding a candidate opens a searchable list, not a <select>: eighty models in a native dropdown are eighty identical lines, so the list carries each candidate's health (dot and word), the measured level and the route, sorts answered routes first, and filters as you type.
  • Switches — the automatic switches that actually happened, kept across sessions: which model failed, what it moved to, whether that recovered the turn, and how long ago. The aggregate is listed first and sorted by how often the model failed, because "which model keeps doing this" is the only form of the answer that can change a chain; the raw events sit below it as evidence. The log is bounded (200 entries) and clearable, since a history nobody can clear eventually describes a machine that no longer exists. Only proven switches are written — the same event that puts a banner on screen — so the count cannot include an intent that never landed. Each raw event names the Session it happened in, and that name is a link: clicking it opens the Session, which is the one place a reader can go and look at what actually failed. The label comes from the shell's own Session list rather than from the log, so a Session renamed since is not remembered under its old name, and a Session the shell no longer lists falls back to the id it was recorded with. Leaving for a Session closes the dialog through the same guard as the close button — an unsaved chain is still not dropped by a stray click, even when the click was somewhere useful.
  • Probes — the last forty probes, newest first, written by every deep probe whether it came from the panel, the picker, the tool or a batch.

The catalog, from the same synthetic renderer:

The model manager on its catalog tab, rendered from the synthetic catalogue

The switch history, same render — one card per incident, each row carrying the clock time it happened at as well as the relative age:

The automatic switch history, rendered from the synthetic catalogue

The dialog's buttons draw their own chrome from the theme (border-l3 hairline, button-primary-fill for the committing action) instead of inheriting the browser's grey <button> face, which reads as a disabled control in this theme.

A model the gateway has withdrawn can be taken out of the catalogue from the row it expands into. This is a local decision, not a gateway call — the plugin cannot unregister a model upstream, and a button claiming otherwise would do nothing — so it is kept in its own overlay document and is undone from the same tab: the removed models are listed underneath the catalogue, marked with whether the gateway still serves them. Removal is keyed by model, not by route and model, because the catalogue rows are model-first: hiding one route's copy would leave the row on screen and the removal would read as broken. A removed model also leaves the fallback chain, because routing a turn into a model the user just took away contradicts the list they edited.

The gateway retires models without telling anyone, so the directory read that decides which rows still exist is worth distrusting on a delay: it is cached for thirty minutes, because each read costs a round trip per route. The header's refresh control forces that read, and the catalog states in one line what it is based on — which route, how many models, how old the reading. The result is reported by route and, when the gateway dropped something, as the names; "3 retired" would hand the matching of ids back to the user, which is the work the control exists to remove. Routes that expose no baseURL have no directory to read at all, so they are left out of that line rather than listed as unavailable: the built-in routes are in that state on every pull, and saying so once per route buried the one reading that meant something under a dozen that could never change and had no action behind them. Only "not one route could be read" turns the pull red.

Only what has no immediate meaning elsewhere waits for Save: the fallback chain and the manual proficiency. Pins, favorites and visibility write the moment you click them, because they already behave that way in the panel and drafting them would make "hide" look broken until a button was pressed. Closing with unsaved edits asks first, and the discard path writes nothing.

Panel controls

Row actions are icon-only buttons with the label on hover and on keyboard focus, so four controls fit beside a model name instead of wrapping the row: pin, favorite, hide, probe. Each carries an aria-label, so it stays readable to a screen reader. The header's two actions (re-check the gateway, deep-probe all) use the same treatment.

Pinning does not move the row

The default order puts pinned and favorite rows first, so re-sorting on every toggle moved the row you had just clicked to the top of the list and took your scroll position with it: scroll down to model 40 of 80, pin it, and you are looking at a different part of the list. Toggling pin, favorite or hide now changes that row's state and not its position.

The order is frozen at the first toggle and applied at the boundaries where you expect the list to settle:

  • collapsing or expanding the list,
  • changing the sort,
  • re-checking the gateway.

While a pending change is outstanding the panel says so, because a frozen list is otherwise indistinguishable from a pin that silently did nothing. Without the freeze the row under the cursor moves; with it, the only thing that moves is the pin icon.

A row carries the decision; the detail block carries the record

A row shows what choosing needs — health dot, name, status, level, recent-form count, and a one-line reason. Everything the Host knows does not fit on one line at a readable size, so each row has a chevron that opens a detail block beneath it. Both the chevron and the model name open it; the chevron is the one that carries aria-expanded and the label, so the block is reachable from the keyboard without adding a second tab stop per row.

The block lists the record in full: status, the complete reason, what checked it and when, whether the route's gateway directory still lists it, level and distance to the next level, a progress bar, uses, the last and previous seven days, heat, first and last use, tokens in/out, and whether the numbers came from the history scan or were counted live.

A hover tooltip was the alternative and it is the weaker one: it needs a pointer, it cannot be read on a touch screen, and it cannot be held open beside a second row to compare.

Density, and why the reason leads

The reason is the most informative thing about a flagged model and the first thing a narrow row truncates. It is rendered first — the source and age trail it, because the status dot and the status chip already say most of what they say. Trailing the reason meant the ellipsis ate exactly the part worth reading.

Two densities are offered, and the choice is remembered:

  • Compact (default) keeps one line per row and gives the reason the remaining space.
  • Comfortable gives the reason a second line, wrapped to two lines before it truncates.

With eighty models, compact is what keeps the list scannable; comfortable is for when you are investigating one of them.

The tooltip is rendered at the panel root rather than inside the row, because the row list is a scroll container that would clip it; it is positioned against the viewport and clamped so a button near an edge cannot push the label off-screen.

Clicking an action updates the row in the same frame (the flag is applied locally first) and the Host's authoritative snapshot then replaces it. A failed request rolls the row back, so the panel never keeps claiming a state the Host refused. Row actions stay visible at reduced opacity instead of appearing only on hover, because a touch device has no hover.

Every mutating action returns the new state under a top-level snapshot; that is the contract that makes the optimistic update safe.

Proficiency and recent use

The panel's top card is Most used: the three models this route is actually worked with, each with its level, a bar to the next level, and the number of uses. Below the counts, a sort control switches the list between the default order, proficiency, and recent use.

Both numbers come from a local ledger, and the distinction between them is the point:

  • Level is earned from the lifetime count and never decreases. It answers "how much have I used this model", which is why it does not fall when you spend a week elsewhere.
  • Heat decays with a 14-day half-life, so it answers "what am I reaching for now". Both Most used sections — the panel's card and the picker's — rank by heat by default, not by the lifetime total, so the oldest model does not win forever.

The level steps are 0, 10, 30, 80, 200, 500 and 1200 uses, giving levels 1–7. Tokens are recorded and displayed but deliberately do not score: a handful of enormous requests should not read as expertise.

The ledger is counted from committed assistant messages in this profile's own session logs. Nothing is sent anywhere and no request is made to build it. Concretely, three choices keep the numbers honest:

  • Counting from committed messages rather than from clicks on the menu means the ledger reflects work actually done. Picking a model you then cancel does not count.
  • A retried request is recorded as an attempt, never as a use. A flaky gateway would otherwise multiply one answer into several levels of proficiency. (This is also why the ledger does not hook the streaming event: that one wraps every call, including retries and replays.)
  • A replayed session event is deduplicated, so resuming a session does not double count it.

The ledger starts empty, but your history does not. On startup the plugin folds the session logs already on disk into the counters, so a model you have been using for months starts at its real level instead of level 1. This runs in the background and is bounded: it scans for up to 20 seconds, yields, and resumes on the next pass, because replaying a large corpus is disk-heavy and must not compete with the live session you are in. Each session it has already counted is marked, so resuming never double counts.

When it finds an empty or partial ledger, the panel says where the pass stands — waiting for the session service, reading your logs, how many responses it counted from how many sessions, or why it stopped. It says so on failure too, deliberately: an earlier version sampled the session service once at startup, and on a host whose service registered a moment later it answered "service not available" and wrote nothing. That empty ledger was indistinguishable from a correct one, and the feature looked dead while reporting nothing at all. Scan history at the bottom of the panel starts a pass on demand.

Because the counters are local and describe your own habits, deleting them does not require hunting for a file: Clear usage at the bottom of the panel, or the reset-usage tool action, for one model, one route, or the whole profile. Set usage.enabled: false to turn the ledger off entirely.

Keeping up with the shipped menu

Replacing a shipped seat means owing it the rows that seat appends. That debt is easy to pay once and then forget: Enable agent teamwork was missing for a week because the replacement simply rendered one fewer line, and nothing failed.

docs/upstream-parity.json is the ledger — every seat we shadow, its child slots, and every upstream row in them, marked as reproduced (with where) or deliberately skipped (with why). tools/check-upstream.mjs reads it and checks three things against the checkout:

  1. each seat and child slot still exists where the ledger says;
  2. every upstream registration into those child slots has a line in the ledger — an unfamiliar id fails the check rather than shipping as a silent omission;
  3. the copy we reproduce still matches the shipped copy, character for character, in both languages.

verify.mjs runs it as one of its checks, so the comparison happens whenever the suite is run instead of whenever someone remembers:

npm run check:upstream          # needs DSH_CHECKOUT

Storage

One atomically-written JSON document per route under $DSH_HOME/storages/llm-manage/<route>.json, holding favorites, pins, hidden, recent and health (status, source, detail, scope, at), plus the usage counters. Deleting the file resets that route's overlay.

Usage is the one part of that document written in batches rather than per event — counters are debounced (usage.flushMs) because a session can commit many messages in a row, and the panel flushes before it reads. The backfill watermark lives in a separate __usage__.json in the same directory.

Two more documents hold decisions that outlive a session: __failover__.json keeps the proven switch history (bounded at 200 entries), and __removed__.json keeps the models taken out of the catalogue (bounded at 400). Both are local to this profile and deleting either one only forgets that history.

Agent tool

The llm_models tool exposes the same operations: snapshot, pin, unpin, favorite, unfavorite, hide, unhide, probe, sync, usage, seed-usage, reset-usage, diagnostics. snapshot and diagnostics need no extra arguments; sync needs route; probe and the overlay actions need both route and model. usage reports the ledger, seed-usage re-runs the historical scan, and reset-usage deletes counters (all of them when given no route). The browser half reads the same data over POST /api/llm-manage.

Applying a change after install

The profile installs this package as a symlink, so file edits land in the running profile immediately. The two halves then behave differently.

index.js (Host half) needs a full harness restart. The host HMR watcher skips node_modules, where the install symlink lives, and its watch root is the profile directory — while the real file resolves outside it. A marker written into index.js was still absent from the running Host's responses minutes later, against a listener process started the previous evening.

client.js (Client half) reloads on its own. The client HMR watcher is a stat poll (default 500 ms) over every row in the composed graph — no node_modules exclusion — and it re-reads the bundle from disk. Touching client.js produces a rebuilt frame on /plugins/events, and the revision the server then advertises matches one computed from the new file's mtime/ctime/size.

Enabling or disabling the bundle does not bust the ESM module cache for either half.

Known limits

  • Deep probing assumes the OpenAI-compatible /chat/completions shape. A route speaking another protocol is reported as probed-with-this-shape, not as protocol-aware.
  • Protocol mismatches are labelled, not fixed. A model's protocol cannot be set per-model: api is a route-level setting. The supported workaround is registering the same baseURL as a second route with api: anthropic-messages — which only helps if your gateway actually serves that endpoint. If it does not, the model is reported as protocol-incompatible rather than silently "fixed".
  • Passive learning needs a real failed request to observe a reason.
  • The usage ledger reads session logs through the host's session service. A host that does not expose it still counts live traffic, but the historical backfill reports itself as unavailable instead of silently reporting zero history.
  • The ledger is per profile, not per project: sessions from different working directories share one set of counters, because the profile is what owns the model configuration.
  • A message that arrives before the plugin is loaded is not counted live. The startup backfill covers it on the next run, unless usage.seed is off.
  • The quota fallback needs the failing model's id, which it takes from the last agent/request it observed for that session.
  • The managed picker (the default) shadows the shipped picker at priority: -1 and re-supplies its inject face. That face is a per-registration business contract, so a future change to the shipped picker's face needs this plugin updated in step. selector.mode: native withdraws the shadow without a restart and leaves the shipped picker in place, decorated by the conversation.input.model.label seat.

Licence

MIT — see LICENSE.

DSH HUB

A community index for DSH plugins. Not an official GitHub or DeepSeek AI product.

CommunityResourcesAPIAbout
—/ 5

No ratings yet

Verified DSH bundle

Commit 9e39a9828f44

Community comments

No comments yet. Be the first to write one.