dsh-plugin-media-gen

Audio and video playback inside the DeepSeek Harness chat, plus xAI Grok Imagine video generation.

The harness already renders images that point at local files; it has no equivalent for video or audio. This plugin fills that gap from the plugin layer:

• show_media — display any existing local image, video or audio file.
• generate_video — generate a clip with xAI Grok Imagine and show it.

Both render into a Turn-level card under the assistant's prose, and also into a keyed tool row, so the player is visible without expanding the collapsible tool group.

Install

ln -sfn "$PWD" ~/.dsh/profiles/desktop/node_modules/dsh-plugin-media-gen

Then, in ~/.dsh/profiles/desktop/package.json, add the link: dependency and dsh-plugin-media-gen to dsh.profile.bundles. Restart the app.

Tools

show_media

Parameter	Type	Notes
path	string	Required. Absolute path to the file.
title	string	Optional caption above the player.

Accepted extensions — image: png jpg jpeg webp gif avif bmp; video: mp4 mov m4v webm mkv; audio: mp3 wav m4a flac ogg opus aac. Anything else is refused with a message that lists the supported set. Returns { path, kind, bytes } and records { path, title? } in presentationMeta, which is where the card reads the path from (no prose parsing).

generate_video

Calls xAI's Imagine API (POST /v1/videos/generations, poll GET /v1/videos/{id}), downloads the MP4 into <DSH_HOME>/media/, and returns its absolute path. Parameters: prompt (required), image_path, duration (1–15 s), aspect_ratio, resolution, model, generate_audio, output_dir.

The API key is resolved per call from GROK_API_KEY, either from the launch environment or from ~/.dsh/.credentials.yaml, so rotating it needs no restart.

Why a custom media route

The harness route GET /api/file serves local files, but it ignores the Range header and answers every request with a full 200 body. A <video> asks for byte ranges, receives a full-body 200, and fires error; audio survives because its pipeline tolerates a full response. This plugin therefore registers its own authenticated route:

GET|HEAD /api/media-gen/file?path=<absolute>

• 200 with Accept-Ranges: bytes and the exact MIME type
• 206 with Content-Range for bytes=a-b, bytes=a- and bytes=-n
• 416 with Content-Range: bytes */<size> for an unsatisfiable range
• streaming instead of buffering the file

The client tries this route first and falls back to /api/file, then to a plain link, so the plugin keeps working if the connection carrier is unavailable.

Two surfaces, one card

Tool calls collapse into a foldable group, so a card registered only in tool.call.toolview is hidden until the group is expanded. The primary surface is the Turn tail: a Turn Definition (media-shown) accumulates the paths that media tools produced, and the card registered in conversation.chat.turnTail draws them under the closing prose — the same seat present and schedule_create use.

Media tool results quote the path (Shown audio: "/…/file.mp3" (2.8 MB)) so a file name containing spaces can still be extracted when the definition replays a session.

Limitations

• The player lives in the Turn card and the tool row. It never appears as a bare assistant block and never from Markdown, because the harness does not expose its text renderer as an extension point.
• generate_video blocks the turn while polling; generation takes one to three minutes and the declared timeout is fifteen.
• Images still render through Markdown; show_media exists mainly for video and audio, where Markdown does not reach.

Development notes

• No @deepseek-ai/* imports: a third-party plugin lives outside the app's bundled node_modules, so those specifiers do not resolve. Tool definitions are the plain objects defineTool would have produced.
• The browser half ships in the harness's window.__ModuleLoader__.load format and requires only react.
• Everything here can be exercised without spending credits: the tools against a scripted fetch, the media route against real files, and the card against a simulated React.

License

MIT