dsh-literature-search

English · 中文

A DeepSeek Harness (dsh) plugin that searches the literature through the official PubMed NCBI E-utilities API and Google Scholar, and returns both backends as one shared paper shape.

• PubMed uses the official API: esearch (find PMIDs) → esummary (metadata) → efetch (abstract / MeSH / keywords) → elink (related work). Free, no key required, documented, and stable over the long run.
• Google Scholar has no official API: with a SerpApi key it uses SerpApi's google_scholar engine; without one it parses scholar.google.com HTML directly. Google throttles and blocks that, so the plugin fails loudly instead of quietly returning an empty result.
• Zero runtime dependencies: only the Node built-in fetch / AbortSignal. The @deepseek-ai/* packages come from the DSH runtime, so there is nothing to build and nothing extra to install.
• Settings page: a dedicated card under Settings → Plugins → Literature Search for NCBI / SerpApi keys, the Scholar backend, rate and result limits, and a one-click connectivity test.

The API research behind these choices — parameters, rate limits, response shapes, measured evidence and rejected alternatives — is written up in docs/API-RESEARCH.md (Chinese).

Tools

Tool	Upstream	What it does
pubmed_search	E-utilities esearch + esummary + efetch	Field-tagged queries with sort, publication-date windows and offset paging. Returns PMID, title, authors, journal, volume/issue/pages, DOI, PMC id, abstract, MeSH and keywords.
pubmed_paper	efetch + esummary	The full record for one PMID, including abstract, MeSH, keywords, publication types and comment/reference lines.
pubmed_related	elink pubmed_pubmed	Ranks articles related to a given one using NCBI's related-article algorithm.
scholar_search	SerpApi / HTML	Google Scholar search: title, authors, venue, year, cited-by count, snippet and PDF link. Supports a year window and paging.
scholar_cite	SerpApi google_scholar_cite	Ready-to-paste citations in MLA, APA, Chicago, Harvard, Vancouver and BibTeX (SerpApi backend only).

All tool names are distinct from the existing dsh-ai4scholar plugin (search_pubmed / search_google_scholar), so the two can coexist.

Install

From npm (recommended)

dsh plugin --profile desktop add dsh-literature-search

dsh plugin forwards to pnpm inside the profile directory and then reconciles the profile's bundle list: any installed dependency whose manifest declares dsh.bundle joins the layer stack automatically. Replace desktop with web if you run the dsh web CLI. Restart DSH afterwards so the new tools load.

Remove it with:

dsh plugin --profile desktop remove dsh-literature-search

From a source checkout (offline)

powershell -ExecutionPolicy Bypass -File install.ps1 -Profile desktop

• -Profile is the target profile (desktop for the desktop app, web for the dsh web CLI).
• The script does two things: it copies the plugin into $DSH_HOME\profiles\<profile>\node_modules\dsh-literature-search, and it merges the literature-search config row into that profile's cordis.patch.yml (idempotent — safe to re-run).
• Restart DSH (desktop app or dsh web) to load the new tools.
• To remove it: powershell -ExecutionPolicy Bypass -File uninstall.ps1 -Profile desktop.

You can also skip the profile entirely and verify retrieval first with the CLI:

node cli.mjs pubmed "CRISPR base editing" --max 3
node cli.mjs paper 33301246
node cli.mjs scholar "base editing" --max 5   # HTML backend without a key; may be blocked

Credentials (optional)

Environment variable / credential	Effect	If unset
NCBI_API_KEY	Raises the E-utilities quota from 3 to 10 requests/second	Still works; the plugin throttles itself to 350 ms per request
SERPAPI_API_KEY	Routes Google Scholar through SerpApi (stable, includes cited-by counts and citation formats)	Falls back to HTML scraping, which Google may answer with HTTP 429
NCBI_EMAIL	CLI only; sent as the E-utilities email parameter	NCBI recommends setting it so they can contact you about unusual traffic

Three ways to configure them. The plugin resolves in this order: inline config → credential service → environment variable.

1. Environment variables in the process that starts DSH.
2. $DSH_HOME\.credentials.yaml (the DSH credential service), written by the settings page, which stores NCBI_API_KEY / SERPAPI_API_KEY.
3. Inline in cordis.patch.yml via pubmedApiKey / scholarSerpApiKey — not recommended, because it is easy to commit by accident.

Settings page

The client plugin (lib/client.js) registers a tab in the settings panel; the server side exposes /plugin/literature-search to back it:

Route	Purpose
GET /plugin/literature-search/config	Returns the (redacted) settings namespace, the revision, and status facts
POST /plugin/literature-search/config	Patch write fenced by expectedRevision; a conflict returns 409
POST /plugin/literature-search/credential	Writes or clears a key for a slot (pubmed / scholar)
POST /plugin/literature-search/test	Actually runs a PubMed esearch and a Scholar search, and reports latency and outcome

What the page gives you:

• Credentials — the NCBI and SerpApi keys are written through the credential service into $DSH_HOME/.credentials.yaml, never into the settings document. The page shows the source (environment = read-only, saved = editable) and the configuration state.
• Backend — scholarProvider is a three-way choice (auto / serpapi / html) and takes effect immediately on save; the client cache is invalidated and rebuilt.
• Rate and results — PubMed/Scholar request intervals, default and maximum result counts, abstract length, timeout and retries all apply immediately.
• Connectivity test — probes PubMed and Google Scholar separately and explains failures, including a "blocked or throttled → configure SerpApi" hint.
• Needs a restart — enabled, pubmedEnabled, scholarEnabled, promptGuidance and promptOrder are registration-time switches; changing them requires restarting DSH. The page says so.

Implementation notes: lib/web.js owns the routes and fact collection; lib/client.js is a hand-written module-loader bundle (window.__ModuleLoader__.load, with require("react") resolved by the DSH client runtime). There is no build step.

How keys are stored, and how their state is decided

Each key slot is judged by two independent probes, and a non-sensitive diagnostic line (ref= / describe= / resolve= / error=) is printed under the badge:

1. The credential service (ctx.credentials, writing $DSH_HOME/.credentials.yaml) — preferred;
2. The in-settings key fields (pubmedApiKey / scholarSerpApiKey, marked role('secret') and redacted on read) — used automatically when the current composition has no credential service. resolveSecret prefers it too.

So whether or not a credential service is present, a key lands somewhere the plugin genuinely reads. If the two probes disagree — describe says unconfigured while resolve returns a value — the page shows "configured" and keeps both raw conclusions in the diagnostic line.

The server also writes the verdict, without any key values, to:

$DSH_HOME/.dsh-literature-search/diagnostics.json

GET /config, POST /credential and POST /test each refresh it. It holds the raw describe/resolve conclusions, error text, DSH_HOME and cwd, and booleans for whether a same-named environment variable exists. If you saved a key and the page still says unconfigured, read this file first.

Both the server and the tool layer read settings / credentials from the injected context (ctx.inject(['credentials'], …)) rather than ctx.get('credentials') — the latter can miss a sibling plugin's service in some compositions, which is the usual reason a key "saves but never resolves".

Configuration

The literature-search row in cordis.patch.yml:

Key	Default	Meaning
enabled	true	Whether the plugin registers its tools
pubmedEnabled	true	Whether the three pubmed_* tools are registered
pubmedBaseUrl	https://eutils.ncbi.nlm.nih.gov/entrez/eutils	E-utilities root
pubmedTool / pubmedEmail	dsh-literature-search / empty	The tool / email parameters sent on every request (required by NCBI's usage policy)
pubmedApiKeyEnv / pubmedApiKey	NCBI_API_KEY / empty	Key reference / inline key
pubmedRateLimitMs	0	Minimum interval between requests; 0 = 350 ms without a key, 110 ms with one
scholarEnabled	true	Whether the two scholar_* tools are registered
scholarProvider	auto	auto (SerpApi when a key exists, else HTML) / serpapi / html
scholarBaseUrl / scholarSerpApiBaseUrl	Official Google / SerpApi URLs	Can point at a self-hosted proxy
scholarSerpApiKeyEnv / scholarSerpApiKey	SERPAPI_API_KEY / empty	Key reference / inline key
scholarHl	en	Google Scholar interface language
scholarRateLimitMs	0	0 = 2500 ms for HTML, 250 ms for SerpApi
defaultMaxResults / maxResultsCap	10 / 50	Count when the model does not specify / the maximum requestable
abstractMaxChars	600	Per-paper abstract or snippet limit; 0 = omit abstracts
requestTimeoutMs / maxRetries / retryBackoffMs	30000 / 3 / 1000	Per-request timeout, retry count and backoff base (429/5xx are retried, honouring Retry-After)
toolTimeoutMs	120000	Budget for one tool call
userAgent	empty (built-in Chrome UA)	Overrides the default UA
promptGuidance / promptOrder	true / 155	Whether the system-prompt paragraph is registered, and its order

Usage examples

On the model side, plain language is enough:

Use pubmed_search for "base editing" reviews published after 2020, take 5, then use pubmed_related on the first one to find 5 related papers.

Use scholar_search to find the 10 most-cited "CRISPR base editing" papers, then pass the first resultId to scholar_cite for BibTeX.

Example output (pubmed_paper 33301246):

PubMed record: Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine.

Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine.
   2020 · The New England journal of medicine · vol 383 · no 27 · pp 2603-2615
   Polack, Fernando P, Thomas, Stephen J, Kitchin, Nicholas et al. (29 authors)
   DOI: 10.1056/NEJMoa2034577 · PMID: 33301246 · PMC: PMC7745181
   https://pubmed.ncbi.nlm.nih.gov/33301246/
   Types: Clinical Trial, Phase III, Journal Article, Randomized Controlled Trial
   MeSH: BNT162 Vaccine, COVID-19, SARS-CoV-2, ...
   Abstract: BACKGROUND: ...

Testing

The offline suite needs two DSH runtime packages (@deepseek-ai/dsh-tools and @deepseek-ai/schemastery). Prepare them either way:

# Route A: extract them from a local DSH install (no network)
powershell -ExecutionPolicy Bypass -File setup-dev-links.ps1   # first run only

# Route B: install them from npm (what CI uses; works without DSH installed)
node tools/fetch-dev-deps.mjs

# Offline: parsers + plugin contract + settings routes + client bundle + config drift
node test/run-all.mjs

# Live: real upstreams, needs access to the corresponding domains
node test/live-pubmed.mjs      # eutils.ncbi.nlm.nih.gov
node test/live-scholar.mjs     # scholar.google.com (a refusal is recorded as SKIP)
node test/probe-scholar.mjs    # connectivity diagnosis: status / result blocks / anti-bot

• setup-dev-links.ps1 calls tools/extract-dev-deps.mjs, which extracts just the DSH runtime closure the tests need (@deepseek-ai/dsh-tools, schemastery, yaml and their transitive deps) out of D:\DSH Desktop\resources\app.asar into .dev-deps/, then junctions it into node_modules/. DSH upgrades and profile layout changes do not affect it. The plugin itself has zero npm dependencies, and neither .dev-deps/ nor node_modules/ is ever copied into a profile.
• tools/fetch-dev-deps.mjs takes the other route: it installs the same packages from npm at the versions pinned in .dev-deps/package.json, then symlinks them into node_modules/. This is what CI uses, on machines with no DSH installed (.github/workflows/ci.yml runs the full offline suite on Ubuntu and Windows across Node 22 and 24).
• The offline suite is 62 cases: MEDLINE parsing 7, Scholar parsing/paging 6, plugin and tools 21 (including "a settings write switches the Scholar backend live", "a credential written through the seam is picked up without a restart" and live result limits), settings routes 14, client bundle 10 and config drift 4. All of them use stub fetch and fake services — no network.
• The client bundle suite drives a stub window.__ModuleLoader__ plus a minimal React shim to actually execute and walk the render tree, which catches typos and null dereferences in the settings page without a browser. It has already paid for itself by finding a state.drafts null crash. Since 0.2.2 it expands function components for a deep render, so nodes owned by child components — such as the credential fields' buttons — are reachable from a test, and it guards the theme regressions directly: primary buttons must use paired tokens, and --dsw-alias-brand-primary must never be a fill colour.
• The live tests record upstream unavailability (no egress, DNS, timeouts, HTTP 429, anti-bot pages) as SKIP rather than FAIL, because that is an environment or policy problem. Only "the page contained result blocks but we parsed zero papers" counts as a failure. Measured here: the Google Scholar HTML backend passed 5/5; PubMed egress from this machine was intermittent and prints SKIP with a reason when it fails.

Repository layout

dsh-literature-search/
├── lib/
│   ├── index.js     # Plugin entry: Config / apply / runtime / settings namespace / prompt paragraph
│   ├── web.js       # /plugin/literature-search routes: config, credentials, connectivity test
│   ├── client.js    # Settings-page client plugin (window.__ModuleLoader__, no build step)
│   ├── http.js      # Rate gate, timeout, retries, Retry-After, JSON/text decoding
│   ├── paper.js     # Shared paper shape, output schema, rendering
│   ├── medline.js   # MEDLINE text parsing (efetch rettype=medline)
│   ├── pubmed.js    # E-utilities client + the three pubmed_* tools
│   └── scholar.js   # Google Scholar (SerpApi / HTML) + the two scholar_* tools
├── tools/
│   ├── extract-dev-deps.mjs   # Extract the test runtime closure from app.asar
│   └── fetch-dev-deps.mjs     # Install the same packages from npm (CI / no DSH)
├── test/            # Offline suites (incl. settings routes and client bundle) + live smoke + fixtures
├── docs/
│   ├── API-RESEARCH.md      # The API research behind these choices (Chinese)
│   └── README.zh-CN.md      # Chinese README — deliberately not in the package root
├── .github/         # CI workflow + issue / PR templates
├── .dev-deps/package.json   # Pins the DSH runtime versions used by the offline suite (the only tracked file there)
├── cli.mjs          # Command-line verification that does not load DSH
├── install.ps1 / uninstall.ps1 / setup-dev-links.ps1
├── cordis.patch.yml
├── LICENSE / CHANGELOG.md / CONTRIBUTING.md / SECURITY.md
└── package.json

Known limitations

• PubMed date filtering uses the print publication date (pdat): the esummary.pubdate of an ahead-of-print record can show a later year. That is PubMed's own behaviour, not a plugin bug.
• Google Scholar has no official API: the HTML backend is intermittent (the same machine can return 200, then 429, then fail to connect). The plugin raises a readable error and names the alternatives. Use a SerpApi key if you need reliability.
• PubMed abstracts may be copyright-protected: NCBI's disclaimer asks users to respect the rights holders' terms. For large-scale mining, download a local copy of PubMed instead.
• scholar_cite only works on the SerpApi backend; the HTML endpoint has no citation-format interface.

When scholar.google.com is unreachable

On a network where scholar.google.com (and the .com.hk / .co.jp / .de mirrors) is blocked, requests fail with UND_ERR_CONNECT_TIMEOUT, while serpapi.com and eutils.ncbi.nlm.nih.gov are fine. The quickest check is the "Test Google Scholar" button on the settings page; from the command line there is a diagnostic script:

node test/probe-scholar.mjs

• resultBlocks > 0 — the page was fetched, so the HTML backend works (scholarBaseUrl can point at any reachable mirror).
• blocked=true — Google served an anti-bot page; slow down or switch to SerpApi.
• ERR UND_ERR_CONNECT_TIMEOUT — the network layer is blocked; pick one of the two options below.

Option A: configure SerpApi (recommended, reliable). Paste the SerpApi key into the "Google Scholar" section of the settings page and save (it goes to $DSH_HOME/.credentials.yaml), or set SERPAPI_API_KEY in the environment that starts DSH. With scholarProvider=auto it switches to SerpApi automatically.

Option B: go through a local proxy. If you run a proxy locally (Clash defaults to 127.0.0.1:7897), start DSH with Node 24's environment-variable proxy support:

set NODE_USE_ENV_PROXY=1
set HTTPS_PROXY=http://127.0.0.1:7897
set HTTP_PROXY=http://127.0.0.1:7897
"D:\DSH Desktop\DSH Desktop.exe"

Note that NODE_USE_ENV_PROXY requires Node 24 and the variables must be set on the DSH process itself, because the plugin's fetch inherits them. If the proxy client is not running (ProxyEnable=0 and nothing listening on the port), this option does nothing.

PubMed is unaffected: eutils.ncbi.nlm.nih.gov is directly reachable on the same network.

Contributing

Issues and pull requests are welcome. Development setup, test commands and code conventions are in CONTRIBUTING.md; the change history is in CHANGELOG.md.

Please do not open a public issue for a security problem — use the private channel described in SECURITY.md. This plugin handles NCBI and SerpApi credentials, so anything that can leak a key is treated as a security issue.

License

MIT