[!NOTE] 📦 This plugin lives in the dsh-plugins monorepo — file issues & pull requests there. npm: @hy-sde-org/dsh-web-search-public

dsh-web-search-public — credential-free web search for DeepSeek Harness

A zero-API-key web search provider for DeepSeek Harness. With a single web_search call it fans the query out to all five public engines in parallel — Startpage, DuckDuckGo, Ecosia, Google, Mojeek — and consolidates the answers by cross-engine consensus, so no single engine's bot challenge, timeout, or empty page can degrade the search. No API keys, no accounts, no fetch provider, no setup.

Identity	Value
Package	@hy-sde-org/dsh-web-search-public
Plugin id	web-search-public
Provider id	public
Engines (fanned out)	startpage, duckduckgo, ecosia, google, mojeek

Based on oh-my-pi — the credential-free engine scrapers are ported from oh-my-pi’s web/search/providers, and the provider is a faithful port of oh-my-pi’s parallel searchPublicWeb aggregate (fan-out, soft/hard deadlines, consensus merge), adapted to the DeepSeek Harness ctx.web seam. oh-my-pi is MIT-licensed (Mario Zechner, Can Bölük); see THIRD-PARTY-NOTICES.md.

Why

DeepSeek Harness ships search providers that need an API key (DeepSeek, Exa, Perplexity). This bundle is the credential-free default: each search is an anonymous request to every public search engine in parallel. A bot-challenged, rate-limited, or down engine is absorbed — the answer comes from whichever engines responded, and results that several engines agree on are ranked first. The whole call is bounded by soft/hard deadlines, so a hung engine can never pin a search to the tool budget.

Prerequisites

• Node.js 22.19 or newer with npm and pnpm on PATH;
• DeepSeek Harness 0.2.0-rc.2 including the standard dsh CLI;
• no API keys — nothing else.

Install the Harness CLI and pnpm before continuing:

npm install --global @deepseek-ai/dsh@0.2.0-rc.2 pnpm
dsh --version

Quick start

Route A — published npm package (recommended)

dsh plugin --profile web add @hy-sde-org/dsh-web-search-public

Route B — from source (validate this checkout or hack on the plugin)

git clone git@github.com:hy-sde/dsh-plugins.git
cd dsh-plugins/dsh-web-search-public
pnpm install
PACKAGE_TARBALL="$(pnpm pack | tail -n 1)"
dsh plugin --profile web add "$PWD/$PACKAGE_TARBALL"
cd ../..

pnpm pack runs the normal prepack build and produces a tarball containing dist/. A direct github:<this-repo> dependency does not contain built output and is not a supported install path — always install the built tarball (or the published package).

Verify the composed configuration

dsh web --dump-config

The composed tree must show web.config.searchProvider: public and a web-search-public plugin row loading @hy-sde-org/dsh-web-search-public.

Run

dsh web

Ask the agent to web_search something. The tool returns rendered sources from the consensus-merged fan-out.

Uninstall

dsh plugin --profile web remove @hy-sde-org/dsh-web-search-public

Already shipped? If your harness checkout already mounts web-search-public in its base bundle (a recent release adopted this package as the shipped default), skip installation — it is already the active provider. Adding this bundle on top would duplicate the loader row and fail at boot (“duplicate loader entry id”).

What the bundle does

The package declares a DSH bundle (dsh.bundle.patch → cordis.patch.yml), so dsh plugin installs it and applies its patch layer to the profile:

1. sets web.config.searchProvider to public (making the credential-free fan-out the default),
2. inserts the web-search-public plugin row.

The keyed DeepSeek route (web-search-deepseek, provider id deepseek-official) is not disabled — deployments that prefer it keep their explicit choice by setting web.searchProvider: deepseek-official and providing the DEEPSEEK_API_KEY credential. No fetch provider is mounted and tool-web keeps fetch: false: the model can search, not fetch arbitrary URLs, which defers SSRF exposure entirely.

How the fan-out works

PublicSearchProvider.search() races engines concurrently (a faithful port of oh-my-pi’s searchPublicWeb):

         ┌────────────────────── fan-out (one query) ──────────────────────┐
         ▼            ▼            ▼            ▼            ▼
     startpage    duckduckgo     ecosia      google       mojeek
         └────────────► consensus merge ◄────────────────┘

• Every engine is asked in parallel. The race returns at the earliest of: all engines settled, the soft deadline (5 s default) with at least one success in hand, or the hard deadline (30 s default) regardless — then aborts every still-running engine. If the soft deadline passes with no success yet and not everything has failed, the call keeps waiting up to the hard deadline for the first success.

• Sources are deduplicated across engines (case/www./trailing-slash-normalized URLs) and ranked by cross-engine consensus — how many engines returned a URL — then best rank, then engine order. The most informative snippet wins; the earlier engine’s title/URL win equal-rank ties.

• Each engine still runs under its own AbortSignal.timeout(timeoutMs) race, so an engine that ignores the aggregate’s cancellation dies at its own cap.

• Rate-limit resilience is best-effort, layered. A hard block (HTTP 4xx) or a zero-result page repeated on consecutive searches opens that engine’s circuit breaker: it is skipped for engineBackoffMs (5 min default, doubling per consecutive trip up to maxEngineBackoffMs), with no request sent, and probed again after the window. Retryable transport failures (HTTP 5xx, timeout, network fetch failure) retry the whole fan-out after an exponential backoff (maxRetries/retryDelayMs, abort-aware sleep); hard blocks are never retried. Once retries are exhausted — or every engine is down — the provider fast-fails for failureCooldownMs (5 min default) with a retry in about Ns error instead of re-blasting engines that just throttled us (which would deepen the block). A pure all-no results aggregate is query-level and is never retried or cooled down.

• available() is true whenever at least one engine is configured — the provider is always usable, which is the point.

• A caller abort (signal.aborted) aborts the fan-out and throws WEB_ABORTED.

• Only if every engine fails does the provider throw WEB_PROVIDER_ERROR with each engine’s reason (id: <reason>; …). Zero organic results, a timed-out engine, and a failed HTTP request all count as “this engine has nothing” and are absorbed by the merge.

The worst case for the default configuration is ≈ hardDeadlineMs (30 s), well under the 60-second search budget dsh-tool-web books in the shipped harness bundle.

Browser escalation (bot walls)

Google, Ecosia and Mojeek are browser-backed (an omp browserFetch port): each engine first tries a plain fetch carrying coherent Chrome navigation headers (client hints derived from its UA), and when that is answered with a bot wall — an enable-JS retry page, a Cloudflare managed challenge ("Ecosia Firewall"), Mojeek's ALTCHA proof-of-work wall, or Google's "unusual traffic" /sorry gate — it escalates to a real browser via ctx.browser.fetchPageHtml (@hy-sde-org/dsh-browser service, stealth launch or CloakBrowser patch backend): home-URL cookie seeding, best-effort ready-selector wait, and ALTCHA auto-solve (force-click of the shadow-DOM checkbox + PoW redirect wait, omp's solveCaptcha). The circuit breaker only opens when every transport — fetch and browser — is blocked.

Escalation is optional: mount the browser service row before this provider and the browser-backed engines light up; without it they stay fetch-only (original behavior), and timeoutMs (default 30 s) is the per-engine budget that covers fetch → browser steps.

Configuration

All options are optional; constants fill the defaults. Configure via the plugin row’s config: in your profile patch, or programmatically via ctx.plugin(...).

Option	Default	Purpose
engines	startpage, duckduckgo, ecosia, google, mojeek	engine ids the fan-out races; this order is the tiebreak for consensus ties; unlisted engines stay disabled
timeoutMs	30000	per-engine transport timeout (ms): bounds one engine even if it ignores aggregate cancellation; browser-backed engines may spend it on fetch → stealth-browser escalation, so the default is 30 s
softDeadlineMs	5000	soft aggregate deadline — return as soon as all engines settled or this passes with ≥1 success
hardDeadlineMs	30000	hard aggregate deadline — return whatever we have, even nothing; must be ≥ softDeadlineMs
maxRetries	1	retries for the all-engines-failed aggregate when at least one engine died from a retryable transport failure (HTTP 5xx, a timeout, or a network fetch failure). Each retry re-runs the whole fan-out after a backoff, so a short engine throttle that strips the first attempt still yields results. 0 disables retrying. Hard blocks (HTTP 4xx) are never retried — they open that engine’s circuit breaker — and a pure all-no results aggregate is query-level and never retried.
retryDelayMs	2000	base delay before retry #1 (ms); doubles on each subsequent attempt.
failureCooldownMs	300000	fast-fail window (ms) after an all-engines-down failure (retry exhausted, or every engine hard-blocked/tripped): searches in this window return a clear retry in about Ns error instead of re-blasting engines that just throttled us (which would deepen the block). 0 disables the window.
engineBackoffMs	300000	base per-engine circuit-breaker window (ms): an engine that hard-blocks (HTTP 4xx) or returns no results on consecutive searches is skipped for this long instead of being re-blasted on every search. Doubles per consecutive trip up to maxEngineBackoffMs; resets after a successful probe. 0 disables the per-engine breaker.
maxEngineBackoffMs	3600000	cap for the doubled per-engine circuit-breaker window (ms).
userAgent	browser-shaped constant	User-Agent sent to the engines (deliberately not the product UA)

- id: web-search-public
  name: '@hy-sde-org/dsh-web-search-public'
  config:
    engines: [startpage, duckduckgo, google, mojeek]   # drop Ecosia, reorder
    timeoutMs: 30000
    softDeadlineMs: 4000
    hardDeadlineMs: 25000
    maxRetries: 1
    retryDelayMs: 2000
    failureCooldownMs: 300000
    engineBackoffMs: 300000
    maxEngineBackoffMs: 3600000

PublicEngineId values are the exported engine ids; any id outside the known set is dropped at startup, and a zero-engine config makes available() false (surfacing WEB_PROVIDER_CONFIGURED_UNAVAILABLE).

Errors

Search failures use the web seam’s WEB_* codes:

Code	Meaning
WEB_PROVIDER_ERROR	every engine failed; message lists id: reason; …

A rate-limited all-failed aggregate (transport signature, retries exhausted) surfaces the same WEB_PROVIDER_ERROR with public search engines are rate limited (…); retry in about Ns. | WEB_ABORTED | caller aborted mid-search | | WEB_PROVIDER_CONFIGURED_UNAVAILABLE | configured provider not available | | WEB_PROVIDER_CONFIGURED_MISSING | configured provider id not registered |

Privacy and security notes

• No credentials. The provider never reads a key, so a leak cannot happen. Engines receive the query, the browser-shaped User-Agent, and nothing else.
• Treat queries as public. Each query is sent to public search engines as an anonymous request — including, because of the fan-out, several engines at once. Do not search for secrets or PII you would not paste into a public search box.
• Amplification is inherent. Every search contacts every configured engine in parallel (up to 5 anonymous scrapes per query). That is the price of resilience and consensus, and it raises bot-challenge / rate-limit exposure compared with a single-engine chain — see docs/operations.md.
• No fetch provider. tool-web fetch: false stays: the model cannot be pointed at arbitrary URLs, so SSRF and unsafe-content surfaces stay closed.
• Redirect policy. All engine fetches use redirect: "error" — redirects are treated as failures and absorbed by the merge rather than leak the caller onto an external location.
• Challenges are expected. Startpage, Ecosia, and Google frequently serve bot checks; the parsers treat challenged pages as zero results and the fan-out tolerates them. See docs/operations.md for runtime expectations.

Compatibility

Component	Supported contract
Node.js	22.19 or newer
DeepSeek Harness	0.2.0-rc.2 (@deepseek-ai/dsh-web, @deepseek-ai/cordis peer range)
Seam	ctx.web WebSearchProvider (no key, no fetch provider)

The provider implements only the WebSearchProvider seam contract; the model-facing tool is @deepseek-ai/dsh-tool-web, which this package does not ship. DeepSeek Harness is a developer preview. Upstream seam-contract changes require a new package release and contract review.

Development

npm ci
npm run check
npm test
npm run build
npm pack --dry-run

The default suite runs parser and provider contract tests against fixture HTML (no network), including the fan-out, consensus merge, and deadline behavior. For live-network validation of the scrapers against current engine markup, run the opt-in real test — it exercises Startpage, DuckDuckGo, and Mojeek for organic results and verifies bot-challenged engines (Ecosia/Google) degrade instead of crashing:

DSH_WEB_SEARCH_REAL_E2E=1 npm run test:real

See CONTRIBUTING.md for the release checklist and design constraints, and docs/operations.md for runtime behavior and troubleshooting.

License and attribution

This package is licensed MIT — the same license as its upstream oh-my-pi. The credential-free engine scrapers and the parallel searchPublicWeb aggregate design are ported from oh-my-pi (MIT License, © Mario Zechner 2025, © Can Bölük 2025-2026); the upstream copyright holders are recorded in LICENSE next to this package’s own notice, and the upstream notice text is reproduced in full in THIRD-PARTY-NOTICES.md.

This plugin is a separate installable package; the harness remains the property of its own project.