Local System One
A local typed-decision control plane for Apple Silicon agents.
让本地小模型负责 Agent 的高频控制决策,让 LLM 专注真正需要生成、规划和深度推理的工作。
Local System One is not another chatbot and does not replace your LLM. It sits before or around an agent's main model and answers small but frequent questions such as:
- Does this task need current Web information?
- Can this request use a cheaper/faster reasoning tier?
- Should this event stay silent, go to a digest, or interrupt the user now?
- Is this task actually complete, or should the Agent continue or verify first?
- Which backend should handle this decision?
- Is a local ANE path healthy enough to use right now?
The project packages a validated Laya Typed Decisions 421M runtime, an optional Apple Neural Engine (ANE) backend, HTTP/MCP interfaces, a reversible Hermes adapter, and a native DeepSeek Harness adapter. The Runtime is the platform-independent decision plane; each Adapter owns how an abstract decision is safely mapped onto its platform.
Published release: v0.2.0 (Runtime 0.2.0, Hermes adapter 0.10.0, DSH adapter 0.7.0). Installation starts OFF. You must start the separate local Runtime and explicitly select Shadow to observe Search decisions; Shadow does not change the Agent's behavior. Production Canary/Active remains unauthorized. See the v0.2.0 scope and Chinese first-use guide.
The repository also retains Phase 8.3 Benchmark Lab source and evidence. Historical H4 executors are frozen and require separate authorization to run. They are not part of the product source bundle or Python wheel; see the integration review.
Why this exists
Large language models are good at generating and reasoning, but agents repeatedly spend those expensive calls on tiny routing decisions.
Local System One turns those decisions into a separate local control layer:
User / Agent
|
v
Local System One
|
+-- hard rules
+-- typed-decision model
+-- policy / health gates
|
v
LLM / tools / notification / routing action
The goal is not "use a small model for everything". The goal is to use a small local model where a structured decision is enough, while keeping the system reversible and fail-open.
What it can decide
| Workflow | Question | Current status |
|---|---|---|
| Search Gate | Does this task need public/current Web information? | Functional. Deterministic hard rules can act; model-probability routing remains conservative/Shadow-first. |
| Model Tier Gate | Can a bounded task use a faster reasoning tier? | Functional, but broad model-based routing remains experimental. |
| Notification Gate | silent / digest / notify_now? | Functional MVP. |
| Completion Gate | complete / continue / verify? | Shadow-only in Phase 8A. Conservative benchmark completed; Active auto-stop is NO-GO. |
| Retry / Escalation Gate | How should a failed tool operation be assessed? | Shadow-only advisory; no automatic retry or model escalation. |
| Choice / Score / Noul | Generic typed decisions for your own control logic | Available through the HTTP API. |
This distinction matters: the project deliberately separates technical capability from what has enough evidence to activate automatically.
Experimental model providers: Model Backend Pilot v1 adds an offline-first, research-only Laya/Jev comparison interface. Default production use remains Laya-only; Jev remote calls and Active Gate execution are not enabled.
Quick start
For a step-by-step explanation, including existing Hermes profiles, mode switching, and how to tell whether Shadow actually loaded in a running Gateway, see the Chinese first-use guide.
Two things to install: (1) the local Python Runtime; (2) the Hermes adapter in the chosen profile. Installing the adapter leaves it OFF. start launches the Runtime; shadow requests Search-only observation; status must confirm an effective Shadow mode before you rely on the observation. No paid main-model calls are required for the local Runtime, but normal Hermes tasks may still use your configured provider.
Requirements
- macOS on Apple Silicon
- Python 3.11–3.13; the hash-locked First-Run dependency set below is for Python 3.12
- Internet access on first model download
From this reviewed source checkout, install into an isolated Python environment:
python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install --require-hashes --only-binary=:all: -r requirements/first-run-macos-arm64-py312.lock
python -m pip install --no-deps .
Choose an existing Hermes profile, or optionally create a new test profile with hermes profile create lsoeval --no-alias --no-skills. In this example, replace lsoeval with the profile you intend to use; skip install when the matching 0.10.0 plugin is already installed:
local-system-one doctor --profile lsoeval
# Only when the pinned model is missing, deliberately download it:
local-system-one download-model --profile lsoeval
# Only when the 0.10.0 adapter is not yet installed in this profile:
local-system-one install --profile lsoeval
local-system-one start --profile lsoeval
local-system-one shadow --profile lsoeval
local-system-one status --profile lsoeval
The download-model and install lines are conditional, not mandatory on every run. If the local-system-one command is missing, first activate the Python environment above. When done, status should show the healthy Runtime, CONFIGURED mode: shadow, and EFFECTIVE mode of registered v0.10 sessions: shadow with Search enabled.
Hermes Gateway caveat: CONFIGURED alone is insufficient. On Hermes 0.23, opening another chat may leave a plugin already loaded as OFF. A profile-scoped refresh may also reconcile shared MCP connections. Follow the first-use guide and do not restart a shared Gateway blindly. The
default source alias is laya-typed-decisions; it resolves and verifies this checkpoint revision:
convaiinnovations/laya-typed-decisions
f9ab0b228f0fc0f14d873dbc99038f135c2da1b2
The plugin is installed under ~/.hermes/profiles/lsoeval/plugins/local-system-one-hermes. The Python package lives in the selected virtual environment; Runtime ownership records live under ~/Library/Application Support/Local System One/profiles/lsoeval. The model is kept in the Hugging Face cache. Failed-start diagnostic logs and a small Runtime lock file can remain in that state directory.
The managed Runtime creates a random bearer credential in the profile's private access-token file with user-only file permissions. Hermes, DSH and MCP use that credential for local requests. Other processes running as the same user, and administrators with sufficient access, can also read it and call the API; loopback alone does not restrict access to Hermes. Clients first verify a fresh Runtime identity challenge, then send the token and task on the same connection. A loaded older plugin needs a reload before it can authenticate. Upgrade older HTTP, MCP and DSH clients with the Runtime; the candidate does not fall back to unauthenticated access. Managed startup refuses a conflicting LOCAL_SYSTEM_ONE_AUTH_TOKEN instead of silently changing the configured credential.
The service accepts loopback listeners only. It limits request bodies to 1 MiB, active connections to 8, concurrent decision work to 1 and queued decision work to 0. A stalled request receives a timeout; its model work may finish in the background, and new work receives an overload response until that slot is free. Socket reads are limited to 5 seconds, request handling to 10 seconds, and each connection to 15 seconds in total.
The Runtime remains offline with respect to model acquisition. To inspect its local service:
local-system-one status --profile lsoeval
Available endpoints include:
POST /v1/choice
POST /v1/score
POST /v1/noul
POST /v1/workflows/search-gate
POST /v1/workflows/model-tier-gate
POST /v1/workflows/notification-gate
POST /v1/workflows/completion-gate
POST /v1/workflows/retry-escalation-gate
GET /health
GET /metrics
The MCP proxy exposes all five Gates and health. Completion and Retry are advisory only. See MCP deployment.
The service binds to loopback by default and does not persist raw task text.
Optional Apple Neural Engine backend
Install the ANE dependencies:
python -m pip install '.[ane]'
The following historical convenience command downloads L512 from Hugging Face. It does not pin the artifact revision or verify its digest; the reproducible v0.2.0 ANE recipe is still pending:
hf download NMTZ/laya-typed-decisions-421m-coreml-ane \
--include 'L512/*' \
--local-dir artifacts/models/laya-typed421-ane
Then start Local System One with the downloaded package:
local-system-one \
--ane-package artifacts/models/laya-typed421-ane/L512/model.mlpackage
The Hugging Face release also includes the validated L192, L384 and L640 research variants. L512 remains the recommended default because it covered 1,966 / 2,000 frozen benchmark decisions while preserving 99.8% selected-decision agreement with the MLX reference.
If you prefer to reproduce the conversion locally, build the same validated fixed-shape L512 package with:
scripts/build_typed421_ane.sh
Then start Local System One with the locally built package:
local-system-one \
--ane-package artifacts/models/typed421-body512-fp16/model.mlpackage
The builder pins laya-coreml to:
4619e0483f07adf39068532e85b42ec2347edb83
The original 421M checkpoint is not duplicated by this project. The converted fixed-shape ANE artifacts are published separately at NMTZ/laya-typed-decisions-421m-coreml-ane, with provenance, SHA256 metadata and benchmark evidence. The Hugging Face packages are ANE transformer-body artifacts and still use the pinned upstream Laya checkpoint for tokenizer, embedding lookup and the host-side action head.
At runtime, the router does not blindly force ANE. It uses token length plus an ANE health gate and falls back to MLX when the accelerator path is unavailable, unhealthy, or unsuitable.
Unified Adapter Contract
Hermes and DeepSeek Harness now implement Adapter Contract v1. The shared contract defines decision semantics, modes, action outcomes, fail-open meaning, and comparable privacy-safe telemetry without standardizing platform internals.
Local System One Runtime
|
v
Decision Contract v1
/ \
Hermes Adapter DSH Adapter
ModelTier=fast is a portable decision. reasoning_effort=low, a provider/model
allowlist, or a concrete Web tool name is not. Those stay inside the relevant
Adapter.
See Adapter Contract v1 and the machine-readable schema. Completion is a backward-compatible Contract v1 extension; see Completion Gate for semantics, bounded context, benchmark evidence, and the Active NO-GO decision.
Hermes integration
First install is OFF. The first Shadow command displays the destination, data categories and effective profile Gate choices before enabling them. Existing explicit Gate settings are preserved:
local-system-one install --profile lsoeval
local-system-one shadow --profile lsoeval
local-system-one status --profile lsoeval
Shadow mode observes decisions without changing the provider request. The configured sidecar receives bounded task text and recent context; other opt-in Gates can send bounded response or failure context. The default address is loopback. Remote HTTPS requires the separate --allow-remote choice, and redirects are refused.
The official Hermes catalog entry was still pinned to 0.6.0 at commit 1095495019b2dff2b4231ed70ec23a3ea716e2f6 when this guide was updated. The published 0.10.0 adapter is available in v0.2.0 Release; the upstream catalog update is reviewed separately. Check which version the catalog currently points to before installing.
Rollback is built in:
local-system-one off --profile lsoeval
local-system-one stop --profile lsoeval --confirm-clients
local-system-one uninstall --profile lsoeval
OFF drains registered 0.10 observations before returning. status reports those live instances separately; older, unregistered plugin sessions remain UNKNOWN and must be reloaded. Stop requires explicit acknowledgement that all clients sharing that Runtime will be affected. After all profiles have been removed, python -m pip uninstall local-system-one removes the shared Python package; model files and Hermes sessions are preserved.
The plugin is designed to fail open: if Local System One is unavailable or a recommendation is incomplete, Hermes keeps its original request unchanged.
See Hermes plugin documentation for the full lifecycle and safety constraints.
DeepSeek Harness integration
The repository includes local-system-one-dsh, a native adapter for the official DeepSeek Harness plugin lifecycle. Main source covers Search, Model Tier, Notification, Completion and Retry / Escalation; the latter three are Shadow-only. Unified Adapter Contract v1 gives both platforms one versioned semantic contract while preserving platform-specific execution and reversible, fail-open behavior.
Validated DSH targets are deliberately exact:
0.1.7-rc.2 @ 477b4f420553e8a52c2fbccc464d7561b239c443
0.2.1-alpha.1 @ 5badb15009ae1756c3afe0ae0cef1faafc290ccc
The adapter supports independent Search, Model Tier, Notification, Completion, and Retry / Escalation observation. Search and Model Tier run at turn start; Notification and Completion observe the completed Agent turn, and Retry can observe an eligible failed tool result. Notification, Completion, and Retry remain Shadow-only. Search Canary retains the audited hard no-Web rules. Model Tier Canary is experimental and disabled by default; when explicitly acknowledged and enabled, only an audited deterministic hard-fast decision on an exact verified provider/model route may copy the first provider request and change reasoningEffort: high -> low. Model/probability decisions never receive active authority.
Historical real provider-wire research included the following paths. Phase 8.4C uses local mock providers and does not validate these remote routes:
DeepSeek Harness -> Nova -> DeepSeek V4.1 Flash
DeepSeek Harness -> StepFun Step Plan -> step-5-preview
The StepFun route completed native high/low requests and an automatic Local System One high-to-low Canary transition, including tool-loop restoration to high after the first provider call.
The installation bundle remains inert in off mode. Connection errors, timeouts, malformed gate responses, unsupported provider/model routes, and any missing Canary authority fail open to native DSH behavior. Turn-scoped Search and Model Tier state is cleared at turn end.
Install the published 0.7.0 bundle from its downloaded Release artifact path:
dsh plugin --profile headless add file:/absolute/path/local-system-one-dsh-0.7.0.tgz
The dsh-plugin distribution branch has been updated to 0.7.0. Production Active and Canary use remain unauthorized.
See DeepSeek Harness adapter documentation, the Phase 0 / Phase 1 probe report, and the Phase 7A Model Tier parity report.
Architecture
Agent / Hermes / DeepSeek Harness / MCP client
|
v
Local System One API
|
v
Decision Engine
/ | \
rules policy typed model
|
v
Router
/ \
MLX L512 ANE
^ |
+-- health/fallback
Main components:
local_system_one/— service, router, health gate, runtime adapters and workflowsintegrations/hermes/— reversible Hermes native pluginintegrations/deepseek-harness/— reversible DeepSeek Harness multi-Gate adapterscripts/— local service, ANE build, launchd and plugin operationsdocs/— design notes and product validationresults/reports/— frozen public benchmark reports
Verified results
The repository grew out of a reproducible JEV/Laya/ANE evaluation program. A few results matter directly to the product:
Typed Decisions quality
On the public 400-case / 2,000-decision benchmark:
| Backend / model | Accuracy |
|---|---|
| Laya Typed Decisions 421M MLX | 0.766 |
| Jev 1.13.0 | 0.737 |
These numbers describe this benchmark only; they are not a general model ranking.
L512 ANE engineering
For the validated 421M fixed L512 path:
- natural benchmark coverage: 1,966 / 2,000 decisions (98.3%)
- selected-decision agreement vs same-subset MLX: 99.8%
- representative gross system energy per decision: about 2.07× better than MLX
- L640 reaches 100% capacity coverage, but was not the best default latency/efficiency tradeoff
The key result is energy-efficient local inference with runtime health gating, not a claim that ANE is universally faster for every request.
Reproducibility
The public release path has been validated from an isolated sanitized bundle:
- clean Python installation
- pinned model acquisition
- pinned
laya-coremlcheckout - fresh L512 conversion
- Runtime startup probe
- old-vs-rebuilt output parity checks
See Public release notes.
Current limitations
This is an alpha control-plane project, not a universal autonomous router.
Important limits:
- The pinned MLX backend clamps the checkpoint's
choice:11+temperature. Treat confidence for Choice requests with 11 or more options as uncalibrated; Search uses the separatenoulprimitive. - Search model probabilities are not trusted as unrestricted final authority.
- Model Tier broad automatic downgrade is not proven to provide a stable latency benefit.
- The ANE backend uses a fixed L512 body and must fall back when requests do not fit or health checks fail.
- Notification preferences are not personalized yet.
- Hermes Canary is intentionally narrow and requires explicit acknowledgement.
- A second physical Mac would strengthen cross-machine reproducibility evidence, although the clean public-path rebuild already passes on the development M4 Mac mini.
- Phase 8.2B's verified native-Hermes/DSH request correlations are bounded localhost-Mock tests; real StepFun post-change billing, Laya/MLX quality, robust overhead and Active behavior have not been requalified.
Conservative defaults are intentional. A wrong routing decision can be more expensive than the small amount of compute it saves.
Documentation
Start here depending on what you want to do:
- Phase 8.4A release baseline and component matrix
- v0.2.0 product scope
- Phase 8.3 archive index
- Phase 8.4B implementation handoff
- Local System One product and API
- Unified Adapter Contract v1
- Phase 8.2B Evidence Contract and privacy-safe native correlation
- Phase 8.2B Budget Ledger and reconciliation
- Phase 8.2B scoped native Mock E2E report
- Search Gate
- Model Tier Gate
- Notification Gate
- MCP and launchd deployment
- Hermes plugin productization
- DeepSeek Harness adapter
- DeepSeek Harness validation report
- DeepSeek Harness Phase 7A Model Tier parity
- Phase 7B Notification + StepFun validation
- Public release / reproducibility
- Upstream provenance
For the underlying experiments:
- Jev Typed Decisions benchmark
- Laya 421M quality benchmark
- Core ML vs MLX parity
- Phase 4 ANE engineering
Project philosophy
System One is useful here as an agent control primitive, not as a replacement for a general-purpose LLM.
The practical pattern is:
hard rule when certainty is available
->
small local probabilistic decision when useful
->
policy / confidence / health gate
->
LLM or action
That makes the control layer cheap, local, inspectable, reversible, and easy to disable.
License
Local System One is released under the Apache License 2.0.
The project depends on and documents upstream work separately. See references/UPSTREAM.md for pinned revisions and provenance.
No comments yet. Be the first to write one.