dsh-model-arena

Run the same prompt through every model you have, and see the difference without squinting.

Install

dsh plugin --profile web add dsh-model-arena

What it does

• Run a prompt through N models. Same prompt, identical parameters, outputs collected side by side.
• Compare. Line-level diff between any two model outputs.
• Score. Auto-scorer (length, latency, reference match) + manual user ratings (1-5).
• Benchmark suites. Save a named set of prompts + models, re-run after changes, get a regression report.
• Panel. Settings page listing recent runs with drill-down to comparison view.

CLI

dsh-model-arena list                              # list recent runs
dsh-model-arena compare <run> <modelA> <modelB>   # diff two models in a run
dsh-model-arena rate <run> --model M --rating 4   # rate a model output

License

MIT