dsh-plugin-trtc-conai

A DeepSeek Harness plugin that packages Tencent RTC Conversational AI (ConAI) as agent tools.

It turns a text agent into a voice agent operator: the model can mint a room, put an AI robot into it, and then hold, steer, or end a live spoken conversation with a human participant. Speech recognition, LLM inference, and speech synthesis all run inside the TRTC pipeline, so audio never passes through the Harness process.

New here? Start with QUICKSTART.md — it walks from zero to a real spoken call, including the client-side step this plugin deliberately does not cover.

What it gives the model

Tool	Purpose	Billed
trtc_issue_credentials	Mint a room plus signed UserSig tickets for a human and a robot. Local signing only.	No
trtc_start_conversation	Put the robot in the room and start a live conversation.	Yes
trtc_stop_conversation	End the task and release resources. Reports an already-stopped task as success.	No
trtc_describe_conversation	Query status (Idle / Preparing / InProgress / Stopped) by task or session id.	No
trtc_update_conversation	Swap voice, model, persona, welcome line, or interruption behaviour mid-call.	No
trtc_speak	Make the robot say an exact line, bypassing the LLM.	No
trtc_invoke_llm	Drive a turn by feeding the LLM content as if the user had spoken it.	No

Every tool returns a structured canonical value, so Code Mode can use it directly:

const { roomId, userId } = await tools.trtc_issue_credentials({})
const { taskId } = await tools.trtc_start_conversation({ roomId, targetUserId: userId })
await tools.trtc_speak({ taskId, text: 'Connecting you to an agent now.', interrupt: true })
await tools.trtc_stop_conversation({ taskId })

Prerequisites

1. A TRTC application with Conversational AI enabled — create one in the TRTC console and pick Conversational AI as the application type.
2. Tencent Cloud API credentials (SecretId / SecretKey) from API key management.
3. The TRTC application's SdkAppId and secret key, used to sign room admission tickets.
4. An LLM API key and, unless you use the built-in flow engine, a TTS key. ConAI proxies these on your behalf.

Install

You need the dsh CLI first (npm i -g @deepseek-ai/dsh), then pick one of three routes. This package is not published to npm yet, so dsh plugin add dsh-plugin-trtc-conai will not resolve — use one of the following instead.

From a git tag or commit (recommended)

dsh plugin --profile <name> add github:Dee3526/dsh-plugin-trtc-conai

A git install fetches sources, not build output. This package ships a prepare script that compiles lib/ at install time, but pnpm ≥10 refuses to run it until you allow it, so the first attempt fails with the exact key to allowlist. Add it to the profile's pnpm-workspace.yaml and re-run:

allowBuilds:
  dsh-plugin-trtc-conai: true

That is permission to execute this package's code on your machine at install time, outside any sandbox the agent runs under. Pin a commit so a later push cannot silently change what runs:

dsh plugin --profile <name> add github:Dee3526/dsh-plugin-trtc-conai#<sha>

From a tarball (no build permission needed)

Prebuilt, so nothing executes at install time:

git clone https://github.com/Dee3526/dsh-plugin-trtc-conai.git
cd dsh-plugin-trtc-conai && npm install && npm pack
dsh plugin --profile <name> add ./dsh-plugin-trtc-conai-0.1.0.tgz

From a local checkout (for development)

git clone https://github.com/Dee3526/dsh-plugin-trtc-conai.git
cd dsh-plugin-trtc-conai && npm install       # `prepare` builds lib/
dsh plugin --profile <name> add ./dsh-plugin-trtc-conai

pnpm links the directory, so edits show up after a rebuild.

Verify it took

dsh --profile <name> --dump-config   # look for a "# == dsh-plugin-trtc-conai" layer

Then boot and ask the agent to list its tools; the seven trtc_* tools should be there. If the layer is missing, the package installed as a plain dependency without its bundle being activated — check that dsh.profile.bundles in the profile's package.json lists dsh-plugin-trtc-conai.

Version compatibility

Declared as peer dependencies, so the plugin binds to whatever your dsh install already has rather than pulling its own copy:

Peer	Range	Verified against
@deepseek-ai/dsh-tools	>=0.0.1-rc.1 <0.2	0.0.1-rc.1 and 0.1.1-rc.2
@deepseek-ai/cordis	>=4.0.1 <5	4.0.2

The full test suite passes on both dsh-tools lines. Note that dsh-tools renamed its registry service between them (ToolRegistry → a default-exported ToolRuntime); that only affects code mounting the registry directly, which the plugin never does — it reaches the registry through ctx.tools.

Configure

Credentials default from the environment, so nothing secret has to be written into a config file:

export TENCENTCLOUD_SECRET_ID=AKID...
export TENCENTCLOUD_SECRET_KEY=...
export TRTC_SDK_APP_ID=1400000001
export TRTC_SDK_SECRET_KEY=...

Missing credentials fail at plugin load with a message naming each one, rather than surfacing later as an opaque signature rejection.

Everything else is a configuration field — nothing tunable is hardcoded. Override the row by id in your profile's cordis.patch.yml. A patch replaces a row's entire config value rather than merging keys, so restate every key the row needs:

- insert:
    - id: trtc-conai
      name: dsh-plugin-trtc-conai
      config:
        region: ap-singapore
        endpoint: trtc.intl.tencentcloudapi.com
        # Use trtc.tencentcloudapi.com for the Chinese mainland site.
        stt:
          language: en
          vadSilenceTime: 1000
          vadLevel: 2          # 2 suppresses far-field noise; higher may swallow single words
        llm:
          model: deepseek-chat
          apiUrl: https://api.deepseek.com/chat/completions
          apiKey: sk-...
          systemPrompt: You are a warm, concise phone assistant. Keep replies under two sentences.
          history: 10
          historyMode: 1       # keep context in sync with played audio
        tts:
          ttsType: flow
          voiceId: v-female-R2s4N9qJ
          language: en
        agent:
          welcomeMessage: Hi, thanks for calling. How can I help?
          maxIdleTime: 60
          interruptMode: 0
          interruptSpeechDuration: 500

Notable options:

• allowStartConversation: false withholds trtc_start_conversation while keeping the read-only and control tools. Use it to let an agent observe and wind down existing calls without ever opening a billable task.
• tts.extra is merged into TTSConfig verbatim, so a provider field this plugin does not model stays reachable without a release.
• roomIdType must match how your client joins: 0 for numeric rooms, 1 for string rooms.
• tts.ttsType switches the emitted key set — flow uses VoiceId/Model, while tencent needs AppId plus its own credential pair.

How a call is wired

trtc_issue_credentials is deliberately separate from trtc_start_conversation. Mint credentials first and let the human client join the room, then start the task — the robot begins pulling targetUserId's stream immediately, and starting first wastes billed time on an empty room.

issue_credentials ──▶ client joins room with (sdkAppId, roomId, userId, userSig)
                          │
                          ▼
                  start_conversation(roomId, targetUserId)  ──▶ taskId
                          │
        ┌─────────────────┼──────────────────┐
        ▼                 ▼                  ▼
     speak()        invoke_llm()      update_conversation()
        └─────────────────┼──────────────────┘
                          ▼
                  stop_conversation(taskId)

The robot identity must be unique within the room. Reusing one interrupts the previous task, so let the plugin generate it unless you have a reason not to.

Operational notes

• Live tasks are cleaned up on unload. Task ids started in a session are tracked, and a plugin unload or config hot-replace issues a best-effort stop so a reload never leaves a billed robot talking to an empty room.
• Stopping is idempotent. An already-stopped task returns alreadyStopped: true rather than an error, because an absent task is the desired end state.
• Vendor errors carry a remedy. FailedOperation.NotAbility becomes "the Conversational AI capability is not enabled on this TRTC application", and the RequestId is always preserved for Tencent Cloud support.
• Cancellation is honored. Each tool forwards exec.signal into the request, and timeoutMs is advertised to the runtime.
• Credentials never reach the model. Secrets live in config and signatures only; the tool schema projection is asserted free of them in the test suite.

Verification

npm test

29 tests, no network or TRTC account required. The signing paths are checked differentially rather than by restating my own arithmetic:

• UserSig is compared byte-for-byte against the official tls-sig-api-v2 package across several identities, with the clock pinned. TRTC uses a non-standard base64 substitution table (+→*, /→-, =→_), and the tests assert no standard-base64 character survives — an RFC 4648 encoder produces tickets the backend silently rejects.
• TC3-HMAC-SHA256 is compared against tencentcloud-sdk-nodejs-intl-en, including a UTF-8 body and both sides of a UTC midnight boundary. The published worked example masks its SecretKey, so its printed signature is unreproducible; the SDK cross-check is the verifiable equivalent. Deriving the credential date from local time is the classic failure here — calls succeed all day and fail only near midnight.

The tool tests load the plugin into a real ToolRegistry and dispatch through the full execution pipeline, covering argument validation, the already-stopped path, error enrichment, and unregistration on disposal.

Test credentials are assembled at runtime rather than written as literals. A string merely shaped like a Tencent Cloud Secret ID trips GitHub's push protection even when it is fake, and the differential design makes the specific value irrelevant — both implementations sign with the same key, so only their agreement is asserted.

Architecture

src/
├── usersig.ts   TLS UserSig signing (node:crypto + node:zlib)
├── tc3.ts       TC3-HMAC-SHA256 signing, split from transport so it is testable
├── client.ts    The five ConAI actions; owns LLMConfig/TTSConfig JSON shaping
├── config.ts    Schemastery configuration schema
├── tools.ts     defineTool definitions, canonical outputs, UI cards
└── index.ts     Plugin entry: credential check, registration, task cleanup

Signing is implemented on node:crypto rather than pulling in tencentcloud-sdk-nodejs, so the runtime dependency set stays at one package and the request path can observe an AbortSignal. The vendor SDKs are devDependencies, used only as test oracles.

License

MIT