Browser & Computer Control (dsh-browser-computer)

English | 中文

Turns "let DSH drive a browser and a computer" into a plugin wired through a single installation. The MCP capabilities are declared by two rows in cordis.patch.yml — playwright-mcp and peekaboo-mcp. That part follows the official "configuration-only bundle" recipe (templates/mcp + references/mcp-bundle.md): MCP wiring alone requires no Host or Client entry file. This package additionally provides Host and Client entries for health checks, repair, and the status UI. See the activation steps below.

Once activated it automatically runs "check environment → install missing dependencies → check permissions → open the permission panels that need attention", and the UI is just a status line plus one Repair button. It never writes profile config (that is the official plugin manager's exclusive job) and never changes system permissions (it only opens the relevant System Settings panel for you to click).

What it wires

Row	Provider	What you get
playwright-mcp	@playwright/mcp	Real-browser control driven by accessibility snapshots and stable refs: navigate, click, fill, tabs, screenshots, network, cookies/storage
peekaboo-mcp	@steipete/peekaboo	Read/write the macOS accessibility tree, screenshots, real mouse and keyboard input (macOS only)

command is resolved at load time via !!js to the absolute npx that sits next to node — no machine-specific path is hard-coded, and it does not rely on the minimal PATH a Finder/Dock-launched app receives (a bare npx cannot be resolved there, so the server dies silently).

Install and activation

Add any of these specs on DSH's official Plugins page. For profiles that allow CLI management, use the commands below. If the Electron App exclusively owns the desktop profile, use its plugin manager instead of trying to manage that profile from the CLI.

# 1) from npm (recommended; specify @version for reproducible installs)
dsh plugin --profile <profile> add dsh-browser-computer

# 2) straight from GitHub
dsh plugin --profile <profile> add github:fegnze/dsh-browser-computer

# 3) local development (points at this directory; refresh/restart requirements still apply)
dsh plugin --profile <profile> add link:<absolute path to this repo>

After installing you must fully quit and reopen the DSH App. Historical measurements:

• The two MCP rows appear in the config layer immediately after install (Config.listConfigs shows playwright-mcp / peekaboo-mcp), but they are not materialised into mcp__playwright__* / mcp__peekaboo__* tools in the running session, and no MCP child process is spawned.
• The Host half's JavaScript is held by the process module cache: without a restart, activation still reports errors from the previous revision.
• The official mcp-bundle recipe itself ends with "install, then call mcp__<serverName>__<tool> to verify".

After the restart, verify by asking the model to call browser_computer_repair, by running the slash command /browser-computer repair, or by calling an MCP tool directly (for example mcp__playwright__browser_navigate).

The Client half (the section on the plugin page) needs one more step: refresh the page. Only after the App restarts does the boot graph include this plugin's client entry, so an already-open page has to be refreshed once to pick it up.

What it does on its own

Plugin activation (install, or App restart) runs repair once, without you clicking anything:

1. Check environment — Node / npx / browser binaries / dependency-package caches
2. Install missing dependencies — npx -y <package> --version warms playwright-mcp and peekaboo-mcp; already-cached packages are never re-downloaded
3. Check permissions — Screen Recording / Accessibility (--no-remote, so the state belongs to the DSH App's own process chain)
4. Guide permissions — if anything is not granted, open the matching System Settings panel (it only opens panels; it never grants anything)

The whole pass is idempotent and runs in the background; a failure in any step only writes one log line and never breaks activation. Set autoRepairOnStart: false to turn it off.

UI: a status line and one button

The UI lives on this plugin's own detail page (official slot plugins.detail.section), not in the host Settings page — this is the plugin's own health check, not a host preference. The Section component returns null for other plugins' subjects (it only renders when subject.kind === 'bundle' && subject.pkg.name === <package name>), so it never leaks onto other plugin pages.

There are exactly two things:

• A one-line status plus a status card (environment / the three permissions / MCP connection), read once automatically when the page opens (read-only, no network)
• One "Repair" button → POST /api/repair: installs missing dependencies, opens the panels that need authorisation, and finally verifies the connection

Both endpoints pass two fences: a custom intent header (which blocks cross-site simple requests) and connection.admit (the Host/Origin fence plus browser authentication). The UI does not require any Harness Client package; controls and styles are its own and reference only --dsw-alias-* theme tokens.

In a previous development environment, a Client change did not need an App restart: editing src/client.js rebuilt the graph and the live page picked it up (the plugin's entry appeared in plugins.detail.section's occupants). Refresh-free updates require an active pnpm run dev:web watcher in the same checkout. Without verifying that watcher, rebuild the affected Web artifacts and refresh the existing page; automatic updates are not promised. Host changes require a full restart.

The two actions

Command	Tool	Side effects	What it does
/browser-computer check	browser_computer_check	none (no network)	local dependencies, macOS permission state, profile wiring conflicts
/browser-computer repair	browser_computer_repair	installs packages, may open panels	install missing dependencies → check and guide permissions → verify the MCP connection (verify: false skips that)

With no argument it runs repair. The tool surface is deliberately just these two: tool schemas are a permanent context cost.

macOS permissions: the pitfall that wastes half an hour

TCC (System Settings → Privacy & Security) has no "Event Synthesizing" switch. That is Peekaboo's name for a capability it derives; it actually hangs off Accessibility:

Name in the report	Where in System Settings	What it affects
Screen Recording	Privacy & Security → Screen Recording	screenshots, window enumeration
Accessibility	Privacy & Security → Accessibility	accessibility-tree reads/writes, background clicks
Event Synthesizing	same place; it opens automatically once Accessibility is granted	synthetic keyboard, foreground pointer actions

Two things you must know:

1. The grant target is the DSH App, not your terminal. The plugin uses --no-remote so Peekaboo reports permissions from the local execution environment, which attributes the state to the DSH App's process chain. Going through a Peekaboo Bridge/daemon instead is a different grant and spawns a resident process.
2. Restart the DSH App after granting. A TCC change does not necessarily take effect for an already-running process.

Security boundaries (read before installing)

• This plugin's computer row passes --allow-foreground: without it, real clicking and typing are impossible. If you only need background accessibility work, delete that single argument from cordis.patch.yml to reduce the privilege.
• Playwright MCP ships browser_run_code_unsafe, which the vendor labels "RCE-equivalent". Page content is untrusted input; do not point it at your daily-driver browser profile first.
• The plugin never touches permissions, so the ceiling is entirely your permission preset.
• Context cost: the two packs together contribute roughly 50 tool schemas to every system prompt. If that is not worth it, delete the corresponding row from cordis.patch.yml.

Do not wire it twice

If you already added MCP rows with the same serverName to ~/.dsh/profiles/<profile>/cordis.patch.yml by hand, they collide with the rows this plugin declares: two MCP instances and duplicate tool names. The check stage reports such conflicts (wiring.conflicts) but will not edit your config — removing the manual row is your call.

Rollback

Remove the plugin on the official Plugins page, or for CLI-managed profiles:

dsh plugin --profile <profile> remove dsh-browser-computer

Removing it retires both MCP rows. To keep the capability but drop the health-check tools, delete just the browser-computer row.

Development

cd dsh-browser-computer
node --test test/*.test.mjs

The tests never run a real npx: subprocess, filesystem and environment are all injected. Worth knowing:

• mcp-probe.test.mjs — runs real newline-delimited JSON-RPC over in-memory streams, covering success / RPC error / timeout / failure to spawn
• doctor.test.mjs — behaviour of check / repair, including "a failure is never dressed up as success"
• http.test.mjs — the route's two fences (intent header + admit), 404, bad JSON, error passthrough
• client.test.mjs — actually executes the Client bundle envelope in a controlled global, which catches a wrong envelope id, a wrong export shape, a wrong slot registration, a style tag that was not tagged with its owner before insertion, or a stray require of another Harness Client package
• host.test.mjs — commands / webServer must be declared through ctx.inject
• consistency.test.mjs — the arguments declared in the patch must match src/packs.mjs's argv, the patch must contain no hard-coded local paths, and the icon must satisfy dsh-app-boot's four rules

Three pitfalls we hit (so you do not)

1. Never use ctx.get('service') during apply. Measured: both ctx.get('webServer') and ctx.get('commands') return undefined while a plugin is applying, so an if (svc?.register) guard silently skips the registration — everything looks fine at install time and you only discover the 404 (or a slash command that does not exist) when you click. Declare optional services with ctx.inject([...], (c) => …); ctx.get is only safe at request-handling time, when the context is fully assembled (for example reading connection).
2. The icon not filling the card is not your problem. The Host's PackageArtwork always renders <img width=size height=size> with CARD_ARTWORK_SIZE = 36 (ROW_ARTWORK_SIZE = 30 for component rows) inside a 48×48 card, so that ring of whitespace is Host layout. Editing the icon file cannot change it.
3. Permission state has three values, not two. In Peekaboo's JSON the permission name lives in the value's name field ({ data: { permissions: [ { name, isRequired, isGranted } ] } }), not in an object key; looking it up by key never matches and reports all three as unknown. And unknown must never be reported as "not granted" — an early revision did exactly that. Now missing contains only denied, while unknown gets its own "cannot read" report. Also, Event Synthesizing is a derived capability (TCC has no switch for it; it follows Accessibility), so it only reports state and never enters missing — otherwise it sends people hunting for a switch that does not exist.

Icon assets

• icon.png — the icon the plugin actually uses: transparent background, only the arrow subject itself, cropped tight to the artwork bounds, 256×256, 47 KB (dsh-app-boot's ceiling is 256 KiB).
• assets/icon-transparent.png — a 768×768 master of the same artwork for reuse or re-export.
• Both are RGBA with alpha 0 in the corners, so they show no white box on dark or light backgrounds. Remember the Host still scales the image to 36×36; that ring is layout, not a white border inside the file.

Known limitations

• The computer pack is macOS-only; on other platforms the report marks it unsupported rather than pretending it works.
• repair pulls packages with npx -y, so the first run can be slow; a timeout is reported explicitly with a "just retry" hint.
• Permission reading depends on the Peekaboo CLI's output. If that format changes, the state degrades to unknown; the report never guesses.
• The health check only verifies "the MCP server connects and its tools are there". It does not verify that a particular action succeeds — that depends on page content and the current foreground window, which are runtime facts.