dsh-observability

English | 中文

OpenTelemetry traces for DeepSeek Harness (dsh): exports each agent session as an OTLP span tree — turn to trace, model step to a child span, tool call to a child of its step — with GenAI semantic-convention attributes, to any collector that accepts OTLP/HTTP.

This is a community plugin (dsh-plugin topic), not part of the official repository. It implements the harness's public telemetry Service Definition (@deepseek-ai/dsh-session-telemetry) as a second Service Provider beside the official one, which exports OTLP logs.

Install

The package ships a cordis.patch.yml, so it installs as a profile bundle:

dsh plugin --profile web add dsh-observability
export DSH_OBSERVABILITY_MODE=FULL
export DSH_OBSERVABILITY_OTLP_URL=http://127.0.0.1:4318/v1/traces

Sharing stays off until DSH_OBSERVABILITY_MODE says otherwise. Configuring an endpoint is not consent to upload session content — see What leaves the machine.

The bundled patch disables the base profile's session-telemetry-otel row: the telemetry Service Definition accepts exactly one backend per context and throws on a duplicate load.

Or mount it as an explicit cordis.yml row:

- id: session-telemetry-otlp-traces
  name: dsh-observability
  config:
    mode: FULL                 # FULL | FEEDBACK_ONLY | DISABLED (default)
    exporter:                  # passed verbatim to the SDK's OTLP/HTTP trace exporter
      url: http://127.0.0.1:4318/v1/traces
      compression: gzip
    processor: {}              # optional; passed verbatim to BatchSpanProcessor
    shutdownTimeoutMillis: 3000
    maxAttributeChars: 32768

Config

Field	Meaning
mode	FULL exports every session live; FEEDBACK_ONLY replays and exports the canonical session log only when the user records feedback; DISABLED (default) constructs nothing and nothing leaves the process. The vocabulary and consent semantics are the Service Definition's, identical to the official provider.
exporter	The complete OTLPExporterNodeConfigBase object, passed verbatim to the OTLP/HTTP trace exporter. url is required outside DISABLED and must be the full traces path (…/v1/traces).
processor	Passed verbatim to BatchSpanProcessor (scheduledDelayMillis, maxQueueSize, maxExportBatchSize, …); batching, retry, and loss policy are the SDK's documented behavior.
shutdownTimeoutMillis	Plugin-owned outer deadline on the SDK's shutdown drain. Defaults to 3000.
maxAttributeChars	Serialized-payload ceiling per span attribute (default 32768). The ceiling bounds the exported value with its …[clipped] marker included, and a clipped span carries dsh.payload_clipped=true — the marker alone cannot be trusted, since a payload may genuinely end in it. The canonical session log keeps the full bytes.

Misconfiguration fails loud at plugin load, before any transport is constructed: a url that is missing, empty, not a string, malformed, or not http(s); a non-positive-integer processor.maxExportBatchSize (the SDK accepts it but then hangs on shutdown); a non-positive or non-finite shutdownTimeoutMillis; a non-positive maxAttributeChars; or an unknown mode.

Field mapping

dsh session event	Span
turn/start / turn/end	trace root span; the end reason lands on dsh.turn.end_reason, and an error reason sets status ERROR with the failure's code: message
step/start / step/end	child span with gen_ai.operation.name=chat
request/header	gen_ai.request.model, gen_ai.provider.name, and the sampling scalars, backfilled onto the step already open. A later header replaces them outright, so a scalar it drops does not survive from the previous one
first assistant/chunk of a step	dsh.step.time_to_first_chunk_ms (time to first token)
assistant/message	assembled reply plus all five gen_ai.usage.* counts (input, output, cache read, cache write, reasoning)
tool/call + tool/result	child span of the requesting step, with gen_ai.operation.name=execute_tool; a failing result sets status ERROR with the reported failure's code: name, when it reports one
user/message	the turn's claimed human prompt (source.kind is user) becomes dsh.turn.input on the trace root. The same event type also carries agent.inject() context and goal continuation rounds, which stay on the timeline as span events rather than replacing the prompt
agent-error ops record	exception span event carrying exception.type / exception.message, plus status ERROR on the open span
every other event type (todo, plan, compaction, hooks, plugin events)	point-in-time span event on the innermost open span
an ops record whose op this package does not recognize	point-in-time span event; the seam owns that op set and may extend it, so an unknown op never truncates the trace

Identifiers are derived, not generated: traceId from (session.id, turn) and spanId from (session.id, turn, step[, callId]). Live capture and FEEDBACK_ONLY canonical-log replay of the same events therefore produce identical trees, and a duplicate handoff after a cursor-less re-adoption lands on the existing span instead of creating a second, disconnected one. Every timestamp comes from the record's own time, never the wall clock, for the same reason.

Spans left open by a missing terminal event are closed by a force-end sweep and marked dsh.force_ended: at the next turn/start with an open predecessor, at the session's shutdown operational record, at the end of a FEEDBACK_ONLY replay, and at backend shutdown.

A missing turn/start — a crash window, or a record a redaction rule withheld — does not orphan the turn's children. The first child recorded under an unopened turn synthesizes that turn's root span, marked dsh.turn.synthesized, so the collector never receives a span whose declared parent it will never be sent.

What leaves the machine

In uploading modes, span attributes carry user and assistant message content, tool arguments and results (command output, file contents), model and usage metadata, and the session cwd (a local path), as returned by the session-telemetry/record waterfall.

This plugin ships no redaction rules. With no waterfall listener mounted, records reach the exporter exactly as captured, so a deployment exporting beyond a trusted boundary mounts its own rules. Provider API keys are structurally absent: adapter credentials are constructor parameters, never session events, so they never enter the session log and therefore never reach telemetry. Serialized payloads are clipped at maxAttributeChars per attribute; the canonical log keeps the full bytes.

Model Experience

None, as this plugin only observes the session stream through the telemetry Service Definition and hands folded spans to the OTel SDK; it never contributes to a model request.

KV Cache effect

None; this plugin neither assembles nor sends a provider request.

Testing

npm test                            # unit: id derivation, folding projection, config fail-loud paths
npm run build && npm run test:e2e   # REAL composition through the Loader against a mock collector

The e2e tier follows the official repository's REAL-composition pattern: the fixture cordis.yml loads the built lib/index.js — the same file a deployment loads — and assertions run against the OTLP payload on the wire, not against internals.

Version compatibility

DeepSeek Harness is in developer preview with no compatibility promises; this plugin pins exact @deepseek-ai/dsh-* versions.

dsh-observability	@deepseek-ai/dsh-*	@deepseek-ai/cordis
0.1.x	0.1.0-rc.6	4.0.1

Known Limitations and Deferred Work

• Best-effort delivery. Inherited from the Service Definition: the handoff cursor marks handed-off, not delivered. Whatever sits in the SDK batch queue at crash time is lost, and a cursor-less re-adoption may re-hand a prefix. Derived identifiers make those repeats idempotent at the receiver, but a durable outbox is out of scope.
• No built-in redaction rules. See What leaves the machine.
• Subagent lineage is not stitched. A forked session's tree starts at its inherited boundary; session.parent_id and session.seed_length ride the span attributes, but no trace links are created.
• One backend per context. Running this plugin and the official OTLP-logs provider simultaneously requires a multi-sink evolution of the upstream Service Definition.
• No metrics signal. The Service Definition's record vocabulary is log-shaped (time, severity, attributes, body), so counters and histograms would have to be derived here rather than captured; only traces are exported.

License

MIT