dsh-plugin-huihui-gemma
Adds Huihui Gemma 4 12B Coder — a local, abliterated Gemma 4 GGUF — as a native model in DeepSeek Harness, with an adapter so both chat and tool calls work.
Category: Models & Reasoning · DSH target: 0.2.2-rc.3 · Platform: Windows, macOS, Linux · License: MIT
What you get
- A
huihui-gemmaprovider and thegemma4-12b-huihui-coder-abliterated-q5model in the model picker of the desktop app, web UI and headless profile. - An OpenAI-compatible adapter that makes this fine-tune usable as an agent model:
- converts its text tool syntax (
<|tool|>call:name{...},<|tool>name(key="value")</|tool>) into OpenAItool_calls; - narrows the harness's 22-tool catalog to four one-argument tools (
run_command,read_file,write_file,list_files) and maps them back to the real harness tools; - removes the harness skill catalog, which otherwise makes this 12B return an empty turn;
- asks the model once for just the shell command when it echoes schema keys (
"command":"command"); - synthesizes SSE, because llama.cpp emits no content deltas for this chat template in streaming mode.
- converts its text tool syntax (
Requirements
- DeepSeek Harness
0.2.2-rc.3(or a compatible release) - Node.js 22 or newer
llama-serverfrom llama.cpp, with the Huihui Gemma 4 12B GGUF and its chat template- Enough memory for the quant you choose (Q5_K_M needs roughly 9–10 GB)
Install
dsh plugin --profile web add github:niko4244/dsh-plugin-huihui-gemma
Or, once the npm package is published:
dsh plugin --profile web add dsh-plugin-huihui-gemma
The bundle configures the llm-pi-ai row for the Huihui route. If the target profile already defines custom providers in that row, merge those entries back after installing (a bundle patch replaces the row's config).
Set an API-key variable before launching dsh (the harness requires one even for a local endpoint):
HUIHUI_API_KEY=local-no-auth
1. Start the model server
llama-server \
--host 127.0.0.1 --port 8766 \
--model Huihui-gemma-4-12B-coder-fable5-v1-abliterated.Q5_K_M.gguf \
--chat-template-file huihui-gemma4-minja-chat-template.jinja --jinja \
--ctx-size 65536 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1 \
--flash-attn on --temp 0.35 --top-p 0.9 \
--alias gemma4-12b-huihui-coder-abliterated-q5
Check it answers: curl http://127.0.0.1:8766/health.
2. Start the adapter
npx huihui-gemma-adapter
It listens on http://127.0.0.1:8769/v1 by default and prints HUIHUI_DSH_PROXY_READY when ready. Leave it running while you use the model.
| Variable | Default | Meaning |
|---|---|---|
HUIHUI_LLAMA_URL |
http://127.0.0.1:8766 |
llama-server base URL |
HUIHUI_ADAPTER_HOST |
127.0.0.1 |
adapter bind host |
HUIHUI_ADAPTER_PORT |
8769 |
adapter port |
HUIHUI_BASE_URL |
http://127.0.0.1:8769/v1 |
URL the provider row points at |
HUIHUI_MODEL |
gemma4-12b-huihui-coder-abliterated-q5 |
model id the adapter serves |
HUIHUI_DEBUG_DIR |
unset | write request/response JSON dumps there |
3. Use it
Restart DeepSeek Harness after installing the bundle, open the model picker and choose Huihui Gemma 4 12B Coder Q5_K_M (local, abliterated). It works in both the Standard and Creator agent presets.
Verified behaviour
> Reply with exactly: HUIHUI_APP_OK
HUIHUI_APP_OK
> Run the command echo HUIHUI_APP_TOOL_9 and report its output.
The output of the command is HUIHUI_APP_TOOL_9.
The harness records a completed pwsh tool call with the real command and reads the result back into the answer.
Notes and limitations
- This fine-tune writes tool calls as text; the adapter parses that text. Tool support is therefore a compatibility layer, not native function calling, and an unusual call format can still be missed.
- Only the four tools above are offered to the model. Other harness tools are still available to the session, but the model will not see them.
- Windows: dsh's workspace ACL sandbox cannot grant write access on exFAT volumes (for example
I:); use a workspace on NTFS or setDSH_PERMISSION_MODE=danger-full-access(which runs tool commands unsandboxed). - The skill catalog is removed only for this provider; other models keep their skills.
- Slow prompt processing is expected: a 5–6k-token harness prompt takes several seconds to prefill on the first turn.
License
MIT
No comments yet. Be the first to write one.