Skip to content

Add VoxCPM engine and LoRA training - #331

Draft
diodiogod wants to merge 2 commits into
mainfrom
codex/add-voxcpm-engine
Draft

Add VoxCPM engine and LoRA training#331
diodiogod wants to merge 2 commits into
mainfrom
codex/add-voxcpm-engine

Conversation

@diodiogod

Copy link
Copy Markdown
Owner

Summary

  • add the official VoxCPM engine with VoxCPM2, VoxCPM1.5, and legacy VoxCPM-0.5B support
  • integrate VoxCPM2 voice cloning, controllable cloning, reference-free Voice Design, TTS Text, and TTS SRT
  • add official-format VoxCPM2 LoRA dataset preparation, configuration, training, checkpoint resume, warm-start, discovery, and inference loading
  • make LoRA adapter identity part of model and audio cache signatures
  • automatically select the CUDA device with the most free VRAM for training, with an explicit device-index override
  • update generated engine documentation and model layouts

Why

This addresses the VoxCPM support requested in issues #99 and #198 while using OpenBMB's official implementation as the runtime and training reference. VoxCPM2 is the recommended model; legacy generations remain available for inference, while integrated training is deliberately limited to VoxCPM2 until legacy compatibility is tested.

Validation

  • Python compilation passed for all changed modules
  • git diff --check passed
  • ComfyUI restarted successfully and registered the new engine/training inputs
  • a real one-step VoxCPM2 LoRA training smoke run completed on an RTX 4090 in about 38.6 seconds
  • the exported lora_weights.safetensors and lora_config.json adapter loaded through the normal VoxCPM engine and generated audio successfully
  • safe checkpoint save/load and optimizer/scheduler resume contract passed

Testing status

This is intentionally a draft because broader manual workflow and regression testing still needs to happen when the maintainer is back at the test machine. No version bump is included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant