DSH HUB
HomePlugin StorePlugin PacksCommunityRankingsResourcesPublish Guide
Plugin source
Back to catalog

zabuxie /

zabuxie/vc-voice-train-skill

Topic repository only

训练 RVC 音色模型:素材体检→切片→特征提取→训练→建索引→验收→部署全流程。修复 RVC 20240604 归一化缺陷。仅用于训练新音色;切换/使用已有音色或 TTS 播放问题无需加载。

★ 17 Stars0 Forks0 IssuesN/A Community rating0 Confirmed installs
View on GitHub
READMESource: main@6b36c2d6

vc-voice-train-skill

Make your AI speak in someone's voice — two routes, pick by need:

  • RVC training — train a model from voice clips; best timbre accuracy, same-language.
  • IndexTTS zero-shot — synthesize straight from a reference clip; the only sane choice for cross-language.

DSH 技能:让 AI 用指定人物的声音说话。两条路线:

  • RVC 训练 —— 用素材训模型,音色准,适合同语种;
  • IndexTTS 零样本 —— 给一段参考音频直接合成,跨语种首选。

Which one? / 怎么选

Need Use Why
Foreign-language source, want Chinese output IndexTTS RVC has no Chinese phonemes → choppy
Source language == target language RVC trained, more accurate timbre
Precise pitch control RVC pitch comes from the driver audio
No training budget IndexTTS one reference clip is enough

Measured on the same source (Japanese CV clips, Chinese output): RVC = choppy · IndexTTS = natural but ~90 Hz too high (fixable via reference selection).

What's inside / 包含什么

rvc-voice-train/
├── SKILL.md                  # full workflow, selection guide, 16 sections
├── scripts/                  # RVC training + acceptance helpers
│   ├── fix_audio_norm.py     # fix RVC 20240604 audio_norm defect (silent noise output)
│   ├── fix_i18n.py           # add missing i18n module (breaks weight export)
│   ├── probe_accept.py       # output-vs-truth acceptance (harmonic purity, bands)
│   ├── pick_pitch_ref.py     # choose reference clips by pitch (median f0 + P90)
│   ├── diagnose_audio.py     # source material quality check
│   ├── verify_env.py         # pre-flight environment check
│   ├── slice_audio.py        # silence-based slicing, keeps source sample rate
│   ├── make_dataset.py       # preprocess WITHOUT re-slicing (preprocess.py loses 76%)
│   ├── check_dataset.py      # dataset integrity + frame alignment
│   ├── make_config.py        # filelist + config.json (40k/32k)
│   ├── make_index.py         # build faiss index (the step CLI training skips!)
│   ├── probe_quality.py      # flat / harmonics / band report
│   ├── patch_train_workers.py# DataLoader workers=0 patch (WinError 1455)
│   ├── hp5_verify.py         # vocal separation purity check
│   └── hp5_batch.sh          # batched UVR5 separation
├── indextts/                 # IndexTTS deployment
│   ├── adapter.py            # FastAPI adapter: dsh-plugin-tts <-> IndexTTS-2.5
│   ├── 启动IndexTTS.cmd      # one-click launcher (frees GPU, checks memory)
│   └── launch_hidden.vbs     # detached launcher (survives .cmd exit)
├── qq/
│   └── qq_voice.mjs          # send synthesized voice to QQ via NapCat
├── patches-index.mjs.diff    # plugin host patch: lang auto-switch + backend control
└── patches-client.js.diff    # plugin UI patch: one-click RVC <-> IndexTTS switch

Why / 为什么需要这个 skill

Both RVC and IndexTTS fail in ways that look like "the model is just bad", with no error:

RVC 20240604 never normalizes audio (audio_norm = audio, normalization commented out). 16-bit PCM hits spectrogram_torch() ~32768× too large → generator latent space is polluted → output is broadband noise. The failure is masked by if loss_mel > 75: loss_mel = 75, so training logs look normal. Detection: loss_disc is 1e8+ instead of 2–6.

IndexTTS has no such defect, but three things bite:

  • Output pitch ≈ reference pitch. Pick a shrill reference and every sentence sounds shrill. Measured: reference 396 Hz → output 441 Hz; reference 313 Hz → output 320 Hz.
  • Random sampling makes tone contours wobble. The 4th tone (去声) slope measured −7.7 / −1.8 / −0.2 across runs — the −0.2 case is audibly "flat". Setting do_sample=False alone does not fix it; you must also pin the RNG seed.
  • On 8 GB cards the built-in low-VRAM chunking splits every 40 characters, which mangles pauses. Raising the threshold to 90 helps; disabling chunking entirely produces a 12.7-second dead tail on long inputs.

Quick start / 快速开始

RVC

<RVC>/runtime/python.exe <skill>/scripts/verify_env.py <RVC>
<RVC>/runtime/python.exe <skill>/scripts/fix_audio_norm.py <RVC>
<RVC>/runtime/python.exe <skill>/scripts/fix_i18n.py <RVC>
# then follow SKILL.md sections 1-4

IndexTTS

# install (see SKILL.md section 10), then:
cp indextts/* <ENV_ROOT>/indextts/
cp -r <voices>/* <ENV_ROOT>/voices/
# double-click 启动IndexTTS.cmd

Point dsh-plugin-tts at provider "Index-TTS2", address http://127.0.0.1:7880.

Plugin patches(可选)

Two unified diffs for @memef1f1y/dsh-plugin-tts@0.6.7:

  • patches-index.mjs.diff — host side:
    • language auto-switch: a zh-CN voice cannot read kana (measured: 0.24 s of near-silence vs 2.6 s correct). Detects kana / hangul / cyrillic / arabic / thai in the text and picks the matching Edge driver voice per request.
    • /dsh-tts-api/backend-control: start / stop either backend, resolve its PID from the listening port, and report lang-patch status.
    • lang-patch-status / lang-patch-restore: DSH reinstalls node_modules on restart, which silently reverts manual patches. These actions detect that and restore from index.mjs.langpatch.bak.
  • patches-client.js.diff — UI: a "快速切换后端" button that stops one backend, starts the other (they cannot share an 8 GB card), switches the provider, waits for readiness, and shows live status + patch health.

Apply from the package root (patch -p1, Git Bash / WSL / patch.exe):

cd <node_modules>/@memef1f1y/dsh-plugin-tts
patch -p1 < <skill>/rvc-voice-train/patches-index.mjs.diff
patch -p1 < <skill>/rvc-voice-train/patches-client.js.diff

# keep a known-good copy so the self-heal action can restore it after a reinstall:
cp lib/index.mjs lib/index.mjs.langpatch.bak

Requirements / 环境要求

Item RVC route IndexTTS route
Bundle RVC 20240604 Nvidia (verified; defect may not exist elsewhere) IndexTTS-2.5
GPU VRAM 32k: ≥8 GB · 40k: ≥12 GB ≥6 GB (8 GB works)
RAM — ≥16 GB (model load uses ~5 GB)
Python 3.9.13 (bundled) 3.10–3.11 only
Disk ~10 GB ~20 GB
Paths ASCII only ASCII only

Key signals / 关键信号

Signal Healthy Broken
RVC loss_disc 2–6 1e8+ → audio_norm defect
RVC loss_mel 10–30, decreasing pinned at 75.000 (hard clamp)
RVC harmonic purity within 0.05 of truth higher → over-smoothed ("电音")
IndexTTS output pitch / P90 close to reference reference too shrill
IndexTTS repeatability 3 runs → identical MD5 seed not pinned

flat (spectral flatness) is not a universal metric. Measured counterexamples: a rejected model had flat 0.040 (lowest) while a user-approved one had 0.21 (over the usual 0.15 threshold). Always compare against ground truth.

Notes / 注意

  • Voice models trained from copyrighted material stay under the original rights holder. This repo ships no audio and no models — methods only.
  • Verified on Windows + RTX 4060 Laptop 8 GB (32k RVC, IndexTTS) and RTX 3060 12 GB (40k RVC).
  • The two backends cannot run simultaneously on an 8 GB card (IndexTTS ~5 GB + RVC ~2–3 GB).

License

MIT for scripts and docs. RVC and IndexTTS follow their own licenses.

—/ 5

No ratings yet

Manifest verification required

Commit 6b36c2d65fed

Community comments

No comments yet. Be the first to write one.

DSH HUB

A community index for DSH plugins. Not an official GitHub or DeepSeek AI product.

CommunityResourcesAPIAbout