@deepseek-ai/dsh-Universal-Vision

中文文档 | English

Universal vision-model plugin for DeepSeek Harness, enabling image description using any configured vision model.

Overview

This plugin provides vision-model capabilities for DeepSeek Harness, allowing the agent to describe images using any vision-capable model. It integrates with the harness's LLM service and attachment system.

Problem Solved

• Provides image description capability using any vision model
• Integrates with DeepSeek Harness's LLM service (ctx.llm)
• Supports credential management through the harness's credential system
• Enables the agent to process and describe images
• Automatically substitutes image blocks with text descriptions when the main model doesn't support images

Implementation

• Uses the harness's LLM service for model communication
• Supports any vision-capable model (DeepSeek Vision, OpenAI Vision, Claude Vision, etc.)
• Registers describe_image tool for explicit image description
• Implements llm/stream waterfall for automatic image substitution
• Configurable through the Settings > Models page

Default Configuration

The default configuration uses deepseek-official, which reuses the DeepSeek API Key configured in Settings > Models.

Custom API Configuration (Recommended)

The default configuration uses DeepSeek, but you can configure other vision models:

Method 1: Modify cordis.patch.yml

- id: universal-vision
  name: '@deepseek-ai/dsh-Universal-Vision'
  config:
    provider: deepseek-official  # or 'openai', 'anthropic', etc.
    model: deepseek-v4-flash  # Optional, override default model

Supported Providers

Provider	Model	Requires API Key
DeepSeek	deepseek-v4-flash	Yes (reuses DeepSeek config)
OpenAI	gpt-4o, gpt-4-vision-preview	Yes
Anthropic	claude-3-opus, claude-3-sonnet	Yes
Any vision model	-	Depends on provider

Configuration Options

Option	Type	Default	Description
provider	string	deepseek-official	Provider type
model	string	Based on provider	Vision model name

Features

describe_image Tool

The plugin registers a describe_image tool that:

• Reads image files (PNG, JPEG, WebP, GIF)
• Saves them as attachments
• Returns detailed text descriptions

Automatic Image Substitution

When the main model doesn't support images:

• Automatically detects image blocks in requests
• Calls the vision model to describe each image
• Replaces image blocks with text descriptions
• Allows text-only models to process image content

Files to Modify in DeepSeek Harness

If installing as a standalone plugin, you need to modify:

1. packages/bundle/base/package.json - Add dependency:

   "@deepseek-ai/dsh-Universal-Vision": "workspace:^"
   

2. packages/bundle/base/cordis.patch.yml - Add plugin entry:

   - id: universal-vision
     name: '@deepseek-ai/dsh-Universal-Vision'
   

3. packages/host/apiproxy/package.json - Add dependency:

   "@deepseek-ai/dsh-Universal-Vision": "workspace:^"
   

4. packages/host/apiproxy/tsconfig.json - Add reference:

   {
     "path": "../../core/universal-vision"
   }