dsh-plugin-auto-vision

DeepSeek Harness (DSH) Smart Multimodal Vision Enhancer Plugin
Resolves image recognition degradation and the "cannot see/recognize images" issue for third-party Claude, GPT-4o, Gemini, and open-source VL models.

---

🌟 Problem Statement

In DeepSeek Harness (DSH), when third-party multimodal models (e.g. claude-3-5-sonnet, gpt-4o, gemini-1.5-pro, qwen2-vl, etc.) are configured via custom providers (OneAPI, NewAPI, CLIProxyAPI, OpenRouter, ModelScope, reverse proxies), users frequently encounter this issue:

When uploading an image, the model replies: "I cannot view images / I cannot recognize images / As an AI I only accept text."
(Ref: GitHub Discussion #112)

🔍 Root Cause in DSH Core

1. Fallback to Text-Only: Unless a model explicitly defines input: [text, image], the pi-ai adapter defaults to DEFAULT_INPUT = ["text"].
2. Silent Image Stripping: DSH's projectImagesForTextModel() intercepts requests when a model lacks the image modality and silently replaces image blocks with:

   [image omitted because this model accepts text only; attachment sha256:xxxx]
   

   The remote model literally never receives the image binary data!
3. Manual Config Pain: Hand-crafting YAML overrides is tedious and easily overwritten by Web UI saves.

---

✨ Features

• 🚀 Zero-Configuration: Works out of the box after installation. No manual YAML editing needed.
• 🧠 Heuristic Vision Rules: Built-in regex patterns covering Claude 3/3.5/3.7/4, GPT-4o/4.5, Gemini, Qwen-VL, GLM-4V, InternVL, Llama 3.2 Vision, and more.
• 🛡️ Runtime Interception: Hooks into ctx.llm and adapter pipelines to inject image input modality dynamically, preventing image stripping.
• 🩺 Diagnostic Tool: Comes with auto_vision_diagnose for instant channel and model capability scans.
• 🎨 Web UI Badge: Displays a Vision Active badge inside the message composer.

---

📦 Installation

Run inside the DSH terminal or system terminal:

dsh plugin install dsh-plugin-auto-vision

Or ask the agent in any active conversation:

"Install the dsh-plugin-auto-vision plugin for me."

---

🛠️ Diagnostics

Ask the agent:

"Run auto_vision_diagnose to check my configured models."

Returns the modality health status of all registered providers and models.

---

⚙️ Configuration Schema

Configurable via Web UI Settings → Plugins → auto-vision or cordis.patch.yml:

Field	Type	Default	Description
enableAutoInjection	boolean	true	Whether to automatically inject image input modality at runtime
customVisionPatterns	string[]	[]	Custom regex strings for matching vision model IDs
excludeModels	string[]	[]	Blacklist of model IDs to exclude from image injection

---

📄 License

Licensed under the MIT License.