dsh-skillflux

Dynamic Skill Runtime Manager for DeepSeek Harness.

SkillFlux keeps the full Skill pool outside the model-facing catalog. For each task, it selects a small set of relevant Skills, mounts them for the current turn, and releases the mounts when the turn ends.

When the local pool has no good match, SkillFlux can search skills.sh and the public GitHub SKILL.md corpus live, then rank candidates with relevance-first quality and 30-day repository activity signals before proposing a pinned mount.

Status: v0.2 for DeepSeek Harness 0.1.1-rc.2. Harness is still a developer preview, so this project follows the current RC API.

中文文档

Why SkillFlux

A growing Skill library should not make every request carry a growing catalog. Large catalogs consume context and make Skill selection less predictable.

SkillFlux acts as a runtime layer between the Agent and its Skill pool. The automatic catalog contains no more than maxActiveSkills selected Skills (three by default), while the official DSH Skill Registry remains the source of truth. Explicit /skill-name invocations remain available separately.

Quick start

You need Node.js 22.20.0 or later and a DeepSeek Harness profile on the 0.1.1-rc.2 package line.

1. Install SkillFlux into the profile you use:

   dsh plugin --profile web add github:YiyuZh/dsh-skillflux
   

   Replace web with another profile name, such as headless, when needed.

2. Restart that Harness profile.

3. Verify the runtime from a DSH conversation:

   /skillflux status
   

For a reproducible deployment, pin a commit:

dsh plugin --profile web add github:YiyuZh/dsh-skillflux#<commit-sha>

The repository includes built lib/ artifacts, so GitHub installation doesn't run a prepare script. The bundle patch disables the official tool-skill consumer and mounts SkillFlux under the unique skillflux loader ID. It keeps the official skill Registry and skill-filesystem provider active.

How it works

User task
  -> ordered rules + deterministic lexical router
  -> optional bounded usage-based tie-breaking
  -> optional embedding fallback for unfilled slots
  -> local Registry + persistent cache + online multi-source discovery
  -> relevance-first quality ranking with 30-day activity signals
  -> select and mount within the skill-count and optional catalog-token budgets
  -> Agent calls a mounted Skill
  -> unmount at turn/end
  -> keep downloaded files until explicit cleanup

A mount is scoped to the receiving Agent. Unmounting removes the runtime registration from future catalogs; it doesn't delete cached files or text already stored in session history.

Capabilities

• Route by ordered rules, then deterministic English and Chinese lexical scores.
• Optionally use successful Skill loads as a small, decaying ranking boost for candidates that already pass the lexical relevance threshold.
• Optionally fill unmatched catalog slots with semantic similarity from a local Ollama or OpenAI-compatible embedding endpoint.
• Limit the model-facing catalog with maxActiveSkills.
• Optionally enforce a conservative estimated-token budget for the Skill catalog prompt.
• Discover candidates from the DSH Registry, the SkillFlux cache, skills.sh, and authenticated GitHub SKILL.md code search.
• Re-rank remote matches by task relevance, marketplace adoption, repository activity, stars, forks, license metadata, and configured trusted owners.
• Resolve remote candidates to immutable GitHub commit SHAs.
• Register cached Skills through the current Agent's ctx.skills scope.
• Verify cached content with a SHA-256 manifest before every load.
• Support per-remote-mount, per-repository/session, and automatic approval policies.
• Expose model tools for loading, searching, and mounting Skills.
• Expose /skillflux commands for status, routing explanations, usage, and cache cleanup.

Routing behavior

SkillFlux routes a task in this order:

1. Preserve an explicit user invocation such as /pdf-reader and exclude that name from automatic routing.
2. Group model-invocable Registry and cached candidates by Skill name. Registry entries represent a name first, while same-name cache entries remain available as fallbacks.
3. Apply matching routes in configuration order.
4. Score the remaining name representatives with deterministic lexical matching and reject scores below minRouteScore.
5. When adaptiveRouting is enabled, add a bounded, time-decaying usage boost only to candidates that already passed minRouteScore. Rules keep priority, and history cannot make an irrelevant candidate cross the threshold.
6. In hybrid mode, use embeddings only when rules and lexical matching leave catalog slots unfilled. Semantic results never displace those earlier matches.
7. Mount the selected names until maxActiveSkills is reached. When catalogTokenBudget is enabled, skip a candidate that would make the estimated catalog prompt exceed that budget. If a candidate can't load, try its same-name fallbacks in candidate-pool order.

The lexical score is:

Match	Score
Complete Skill name or its space-separated form	+100
Each matching name token	+20
Each matching whenToUse token	+8
Each matching description token	+3

For equal scores, Registry candidates rank before cached candidates, and cached candidates rank before remote candidates. Cache ties prefer higher install counts and then a stable source/name order.

Semantic fallback uses cosine similarity, filters results below minEmbeddingSimilarity, and reports a rounded 0-100 score. It runs only after the ordered rule and lexical stages.

Online quality discovery

Remote discovery is live, not a bundled catalog:

1. skills.sh supplies marketplace matches and install counts.
2. When GITHUB_TOKEN or GH_TOKEN is available, GitHub Code Search finds matching public SKILL.md files outside the marketplace. SkillFlux fetches and validates each matched frontmatter before accepting it.
3. GitHub repository metadata supplies the immutable HEAD commit, stars, forks, license, archive state, owner type, and last push time.
4. SkillFlux rejects zero-relevance, archived, disabled, below-star, and below-quality candidates, then returns the highest-quality matches.

The quality score is capped at 100. Relevance is a gate and the largest single component, so a famous but unrelated repository cannot outrank an exact new match merely because it has more stars.

Signal	Maximum contribution
Task/name/description relevance	55
skills.sh installs	15
GitHub stars	15
GitHub forks	5
Repository activity, with the configured recent window worth most	10
Trusted owner, organization ownership, and license metadata	15

remoteRecentActivityDays defaults to 30. Activity inside that window receives the full freshness contribution; older maintained projects decay gradually instead of being discarded. Add owners you have independently vetted to remoteTrustedOwners; being an organization or having many stars is not itself treated as verification.

GitHub code search requires authentication. Start DSH from a shell that exposes one of the standard variables, for example in PowerShell:

$env:GH_TOKEN = gh auth token
dsh web

Without a token, SkillFlux continues to search skills.sh and enrich those results through the public GitHub REST API. It simply skips the broader GitHub code-search provider.

Configuration

SkillFlux accepts these plugin options:

maxActiveSkills: 3
minRouteScore: 8
approvalPolicy: always       # always | session | automatic
remoteDiscovery: automatic   # automatic | on-demand | off
remoteProviders: [skills.sh, github]
remoteSearchLimit: 5
remoteSearchTimeoutMs: 30000
remoteMinQualityScore: 35     # 0-100
remoteMinStars: 0
remoteRecentActivityDays: 30
remoteTrustedOwners: []       # e.g. [anthropics, openai, vercel-labs]
catalogDescriptionMaxLength: 160
catalogTokenBudget: 0              # 0 disables; otherwise 64-1000000
maxSkillFiles: 1000
maxSkillBytes: 10485760
installTimeoutMs: 300000
routerMode: lexical                # lexical | hybrid
embeddingProvider: ollama          # ollama | openai-compatible
embeddingEndpoint: http://127.0.0.1:11434/api/embed
embeddingModel: embeddinggemma
embeddingApiKeyEnv: SKILLFLUX_EMBEDDING_API_KEY
embeddingTimeoutMs: 5000
embeddingCandidateLimit: 128
embeddingCacheSize: 512
minEmbeddingSimilarity: 0.45
usageTracking: true
usageMaxEntries: 1000
adaptiveRouting: false
adaptiveMaxBoost: 6
adaptiveMinUses: 2
adaptiveHalfLifeDays: 30
routes: []

Add ordered rules when a known task must prefer specific Skills:

routes:
  - matchAll: [pdf, analyze]
    skills: [pdf-reader, document-parser]
  - matchAny: [react, frontend]
    skills: [react-specialist]

A rule can contain matchAll, matchAny, or both. Rule results keep their declared order, skip unavailable Skills, and still respect maxActiveSkills.

Hybrid embedding router

Embedding routing is opt-in. The recommended private setup uses Ollama:

ollama pull embeddinggemma

routerMode: hybrid
embeddingProvider: ollama
embeddingEndpoint: http://127.0.0.1:11434/api/embed
embeddingModel: embeddinggemma

For an OpenAI-compatible embedding service, select the protocol and point at its exact embeddings endpoint:

routerMode: hybrid
embeddingProvider: openai-compatible
embeddingEndpoint: https://provider.example/v1/embeddings
embeddingModel: provider-embedding-model
embeddingApiKeyEnv: SKILLFLUX_EMBEDDING_API_KEY

Set the named environment variable in the process that launches DSH. SkillFlux never stores that value. Candidate vectors are kept in a bounded in-memory LRU and disappear when the plugin stops. An unavailable, malformed, timed-out, or reconfigured embedding endpoint fails open to the lexical result.

Usage statistics and adaptive routing

Usage tracking records successful mounts and successful skill({ name }) loads by exact candidate ID. It is enabled by default, while adaptive routing is opt-in:

usageTracking: true
adaptiveRouting: true
adaptiveMaxBoost: 6
adaptiveMinUses: 2
adaptiveHalfLifeDays: 30

Records are stored atomically in $DSH_HOME/storages/skillflux/usage.json (normally ~/.dsh/storages/skillflux/usage.json) and bounded by usageMaxEntries. The file also has a hard 2 MiB limit; least-recently-useful records are evicted first when either bound is reached. SkillFlux stores only the candidate ID, Skill name, origin, source, counters, and timestamps. It does not store task text, Skill instructions, or resources.

The boost is capped by adaptiveMaxBoost, requires at least adaptiveMinUses successful loads, and halves after adaptiveHalfLifeDays without use. A telemetry read or write failure falls open to normal routing. Set usageTracking: false to disable persistence; in that case adaptiveRouting must also remain false.

Catalog context budget

catalogTokenBudget is an optional second bound in addition to maxActiveSkills. The default 0 keeps existing behavior. A nonzero value preflights each mount and rejects only the candidate that would exceed the budget, allowing later smaller or same-name fallback candidates to continue:

maxActiveSkills: 3
catalogDescriptionMaxLength: 160
catalogTokenBudget: 512

The estimate covers the complete replacement-form Skill catalog prompt after description truncation. It uses ceil(UTF-8 bytes / 3): intentionally conservative for typical English and close to one token per CJK character, but it is not a model-specific tokenizer result. /skillflux status reports the current estimate and /skillflux explain marks rejected candidates as budget-skipped.

Approval policies

Policy	Behavior
always	Request native DSH approval for every remote mount. This is the default.
session	Request approval for the first successful install from a repository, then trust that repository for the current session.
automatic	Download and mount the highest-ranked remote candidate without approval. Use only in a trusted environment.

If approval is unavailable, rejected, or canceled, the remote mount fails closed.

Model tools and user commands

The model can use:

• skill({ name }) to load instructions for a Skill already mounted this turn.
• skillflux_search({ query, remote? }) to search installed, cached, and immutable remote candidates.
• skillflux_mount({ candidateId }) to mount a candidate from the current SkillFlux discovery state.

You can use:

/skillflux status
/skillflux explain
/skillflux usage
/skillflux cache list
/skillflux cache clean <cache-id>
/skillflux cache clean all

explain shows each candidate's router stage, score, base score, adaptive boost, and whether it was selected, successfully mounted, or skipped by the catalog budget.

Cleanup skips cache entries that are still mounted. At turn/end, SkillFlux unregisters runtime mounts but retains downloaded files for later reuse.

Security and trust

• Remote discovery accepts only public GitHub repositories from skills.sh or authenticated GitHub SKILL.md code search.
• GitHub-discovered SKILL.md files are bounded to 256 KiB and must pass the same supported frontmatter parser before they become candidates.
• The SHA-256 hash of a GitHub search preview must match the SKILL.md selected by the installer, preventing a same-name Skill elsewhere in the repository from silently replacing the reviewed match.
• Archived and disabled repositories are rejected. Repository popularity, activity, and license metadata are ranking evidence, not a security verdict.
• Each remote result is resolved to a 40-character commit SHA before SkillFlux creates its candidate ID.
• Installation downloads that immutable GitHub codeload archive through the pinned skills@1.5.23 CLI.
• Transport extraction is capped at 5,000 files. The selected Skill is separately capped at 1,000 files and 10 MiB by default.
• SkillFlux checks paths, symlinks, frontmatter, file counts, byte counts, and a SHA-256 content manifest before mounting.
• SkillFlux caches scripts as resources but never executes them.
• Automatic discovery sends a bounded keyword query instead of the complete user message.
• Hybrid routing sends at most 1,000 characters of the direct task and at most 1,000 characters of each candidate's name, whenToUse, and description to the configured embedding endpoint. It never sends Skill bodies or resources.
• Usage records never include task text or Skill content and are bounded to usageMaxEntries entries in the DSH storage directory.
• GITHUB_TOKEN or GH_TOKEN is optional. It enables broad GitHub code search and batched repository enrichment; SkillFlux doesn't persist it.

Skills are external instructions and can be malicious. Approval is a trust decision, not a sandbox. Keep DSH permissions, sandboxing, and tool approvals enabled.

Evaluation

Run the versioned routing corpus without an API key or network access:

corepack pnpm eval

The suite contains 36 lexical cases, 4 adaptive safety cases, 8 provider-independent semantic-vector cases, 7 catalog-budget cases, and 8 remote-quality pairwise cases covering English, Chinese, normalization, rules, thresholds, capacity, ranking, deduplication, semantic top-k, context budgets, freshness, trust, adoption, and negative rejection.

Metric	Current baseline
Exact ordered match	100.0%
Top-1 accuracy on positive cases	100.0%
Negative-task rejection	100.0%
Selector-limit compliance	100.0%
Semantic exact ordered match	100.0%
Semantic positive Top-1	100.0%
Semantic negative rejection	100.0%
Remote-quality pairwise ordering	100.0%

These results verify the deterministic router and vector-ranking contracts against checked-in inputs. The semantic vectors are synthetic, so these results do not measure a particular embedding model, third-party Skill quality, or the final answer from an online model. Read the evaluation corpus guide for the case format and limitations.

Known limitations

• Hybrid quality depends on the configured embedding model. SkillFlux does not download or manage that model.
• Semantic fallback considers at most embeddingCandidateLimit local candidates in current Registry/cache order.
• Catalog token counts are portable estimates, not exact counts from the configured chat model. They exclude loaded Skill bodies, tool schemas, and other session history.
• GitHub Code Search is unavailable without GITHUB_TOKEN or GH_TOKEN; the skills.sh provider remains available.
• Quality scoring is evidence-based triage, not a code audit. Inspect the exact pinned candidate and keep approval/sandbox controls enabled before mounting.
• Unmounting can't remove text already committed to session history.
• A new upstream commit creates a new immutable cache entry. Old entries remain until explicit cleanup.

Development

External contributors should follow CONTRIBUTING.md for the fork, topic-branch, offline quality-gate, live GitHub discovery test, evaluation, and pull-request review workflow.

corepack pnpm install
corepack pnpm check
corepack pnpm eval
corepack pnpm test:discovery-live
corepack pnpm test:embedding-live
corepack pnpm pack --dry-run

test:discovery-live runs a real skills.sh query and also GitHub Code Search when GITHUB_TOKEN or GH_TOKEN is present. Override the task with SKILLFLUX_DISCOVERY_QUERY, add comma-separated trusted owners with SKILLFLUX_TRUSTED_OWNERS, or set SKILLFLUX_REQUIRE_GITHUB=1 to fail when the GitHub provider is unavailable.

test:embedding-live expects the configured Ollama model to exist. Override the defaults with SKILLFLUX_EMBEDDING_MODEL, SKILLFLUX_EMBEDDING_ENDPOINT, and SKILLFLUX_EMBEDDING_PROVIDER when testing another endpoint.

The test suite covers routing, DSH catalog virtualization, explicit invocation, remote response validation, cache integrity, lifecycle cleanup, bounded usage storage, adaptive-threshold safety, and the bundle patch. Read CONTRIBUTING.md before submitting a change.

License

MIT