DSH HUB
HomePlugin StorePlugin PacksCommunityRankingsResourcesPublish Guide
Plugin source
Back to catalog

niko4244 /

niko4244/dsh-plugin-huihui-gemma

Verified

Huihui Gemma 4 12B (local llama.cpp) provider for DeepSeek Harness, with a chat and tool-call compatibility adapter.

★ 1 Stars0 Forks0 IssuesN/A Community rating0 Confirmed installs
View on GitHub
READMESource: main@b69c998e

dsh-plugin-huihui-gemma

Adds Huihui Gemma 4 12B Coder — a local, abliterated Gemma 4 GGUF — as a native model in DeepSeek Harness, with an adapter so both chat and tool calls work.

Category: Models & Reasoning · DSH target: 0.2.2-rc.3 · Platform: Windows, macOS, Linux · License: MIT

What you get

  • A huihui-gemma provider and the gemma4-12b-huihui-coder-abliterated-q5 model in the model picker of the desktop app, web UI and headless profile.
  • An OpenAI-compatible adapter that makes this fine-tune usable as an agent model:
    • converts its text tool syntax (<|tool|>call:name{...}, <|tool>name(key="value")</|tool>) into OpenAI tool_calls;
    • narrows the harness's 22-tool catalog to four one-argument tools (run_command, read_file, write_file, list_files) and maps them back to the real harness tools;
    • removes the harness skill catalog, which otherwise makes this 12B return an empty turn;
    • asks the model once for just the shell command when it echoes schema keys ("command":"command");
    • synthesizes SSE, because llama.cpp emits no content deltas for this chat template in streaming mode.

Requirements

  • DeepSeek Harness 0.2.2-rc.3 (or a compatible release)
  • Node.js 22 or newer
  • llama-server from llama.cpp, with the Huihui Gemma 4 12B GGUF and its chat template
  • Enough memory for the quant you choose (Q5_K_M needs roughly 9–10 GB)

Install

dsh plugin --profile web add github:niko4244/dsh-plugin-huihui-gemma

Or, once the npm package is published:

dsh plugin --profile web add dsh-plugin-huihui-gemma

The bundle configures the llm-pi-ai row for the Huihui route. If the target profile already defines custom providers in that row, merge those entries back after installing (a bundle patch replaces the row's config).

Set an API-key variable before launching dsh (the harness requires one even for a local endpoint):

HUIHUI_API_KEY=local-no-auth

1. Start the model server

llama-server \
  --host 127.0.0.1 --port 8766 \
  --model Huihui-gemma-4-12B-coder-fable5-v1-abliterated.Q5_K_M.gguf \
  --chat-template-file huihui-gemma4-minja-chat-template.jinja --jinja \
  --ctx-size 65536 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1 \
  --flash-attn on --temp 0.35 --top-p 0.9 \
  --alias gemma4-12b-huihui-coder-abliterated-q5

Check it answers: curl http://127.0.0.1:8766/health.

2. Start the adapter

npx huihui-gemma-adapter

It listens on http://127.0.0.1:8769/v1 by default and prints HUIHUI_DSH_PROXY_READY when ready. Leave it running while you use the model.

Variable Default Meaning
HUIHUI_LLAMA_URL http://127.0.0.1:8766 llama-server base URL
HUIHUI_ADAPTER_HOST 127.0.0.1 adapter bind host
HUIHUI_ADAPTER_PORT 8769 adapter port
HUIHUI_BASE_URL http://127.0.0.1:8769/v1 URL the provider row points at
HUIHUI_MODEL gemma4-12b-huihui-coder-abliterated-q5 model id the adapter serves
HUIHUI_DEBUG_DIR unset write request/response JSON dumps there

3. Use it

Restart DeepSeek Harness after installing the bundle, open the model picker and choose Huihui Gemma 4 12B Coder Q5_K_M (local, abliterated). It works in both the Standard and Creator agent presets.

Verified behaviour

> Reply with exactly: HUIHUI_APP_OK
HUIHUI_APP_OK

> Run the command echo HUIHUI_APP_TOOL_9 and report its output.
The output of the command is HUIHUI_APP_TOOL_9.

The harness records a completed pwsh tool call with the real command and reads the result back into the answer.

Notes and limitations

  • This fine-tune writes tool calls as text; the adapter parses that text. Tool support is therefore a compatibility layer, not native function calling, and an unusual call format can still be missed.
  • Only the four tools above are offered to the model. Other harness tools are still available to the session, but the model will not see them.
  • Windows: dsh's workspace ACL sandbox cannot grant write access on exFAT volumes (for example I:); use a workspace on NTFS or set DSH_PERMISSION_MODE=danger-full-access (which runs tool commands unsandboxed).
  • The skill catalog is removed only for this provider; other models keep their skills.
  • Slow prompt processing is expected: a 5–6k-token harness prompt takes several seconds to prefill on the first turn.

License

MIT

—/ 5

No ratings yet

Verified DSH bundle

Commit b69c998e9e24

Community comments

No comments yet. Be the first to write one.

DSH HUB

A community index for DSH plugins. Not an official GitHub or DeepSeek AI product.

CommunityResourcesAPIAbout