Skip to content

Commit 42d3217

Browse files
crispasr integrationclaude
andcommitted
docs: update readme and plan for v0.4.0
readme.md: - 31+ handlers (was 28+), 10 CrispASR backends (was 7) - Document WavMark (MIT) as preferred neural watermark - Document FLAC/Opus metadata support - Document voice-cloning API consent gate (i_have_rights field, 403) - Document persistent audit log path - Document CrispASR kokoro as spoken disclaimer source - Update ecosystem table, project structure, test count (212) - Add optional dep install commands - Add voice cloning curl example PLAN.md: - Add Phase 6: EU AI Act Art. 50 compliance (v0.4.0) - Document all sub-tasks with commit refs and CI results Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 10becea commit 42d3217

2 files changed

Lines changed: 135 additions & 24 deletions

File tree

PLAN.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -183,6 +183,60 @@ All 5 phases implemented, tested, and pushed.
183183
| 4 | `1b4aee0` | 48 pass |
184184
| 5 | `0f734d2` | 51 pass |
185185

186+
---
187+
188+
## Phase 6: EU AI Act Art. 50 Compliance (v0.4.0)
189+
190+
Implemented 2026-06-22. Deadline: 2026-08-02.
191+
192+
### 6.1 Watermark embedding on all outputs
193+
194+
The critical gap: `watermark_embed()`, `inject_wav_metadata()`, `inject_mp3_metadata()`,
195+
and `c2pa_sign_file()` were defined in `watermark.py` but never called on synthesized audio.
196+
197+
- **main.py**: Post-synthesis watermark + metadata injection for all handlers
198+
- CrispASR handlers skipped (binary already watermarks)
199+
- WAV/MP3/FLAC/Opus metadata injection on all outputs
200+
- C2PA signing if cert/key configured
201+
- Same pipeline in `test_all_models()` loop
202+
- **server.py**: Same watermark pipeline for API responses
203+
- `X-CrispTTS-Watermarked: true` response header
204+
205+
### 6.2 WavMark neural watermark (MIT license)
206+
207+
Added as preferred neural backend over AudioSeal (CC-BY-NC model weights).
208+
209+
- **watermark.py**: `load_wavmark()`, `_embed_wavmark()`, `_detect_wavmark()`
210+
- Fixed 16-bit "CT" payload for CrispTTS detection
211+
- Sample-rate aware (resamples to 16 kHz for WavMark, applies delta at native rate)
212+
- Priority: WavMark (MIT) > AudioSeal (Python) > CrispASR GGUF > spread-spectrum
213+
214+
### 6.3 New CrispASR TTS backends
215+
216+
- `crispasr_f5_tts` — F5-TTS flow-matching, 24 kHz, Apache 2.0
217+
- `crispasr_melotts` — MeloTTS VITS2, 44.1 kHz, MIT
218+
- `crispasr_piper` — Piper VITS via C++, 250+ voices, 30+ langs
219+
220+
### 6.4 Voice-cloning safety
221+
222+
- **Server consent gate**: `"i_have_rights": true` required in API request body for cloning models (returns 403 otherwise)
223+
- **Expanded detection**: CrispASR cloning backends (`vibevoice`, `indextts`, `voxcpm2`, `qwen3_tts`) added to keyword set
224+
- **Persistent audit log**: `~/.cache/crisptts/consent_audit.log` (not just stderr)
225+
- **Spoken disclaimer**: CrispASR kokoro (local, first) > Edge TTS (cloud) > beep
226+
227+
### 6.5 FLAC/Opus metadata
228+
229+
- `inject_flac_metadata()` — Vorbis comments via mutagen
230+
- `inject_opus_metadata()` — OggOpus tags via mutagen
231+
- Wired into main.py and server.py
232+
233+
| Commit | Tests | CI |
234+
|--------|-------|----|
235+
| `01b4d41` | 199 pass | py3.10/3.11/3.12 + ruff ✓ |
236+
| `10becea` | 212 pass | py3.10/3.11/3.12 + ruff ✓ |
237+
238+
Released as [v0.4.0](https://github.com/CrispStrobe/CrispTTS/releases/tag/v0.4.0).
239+
186240
### Live test results (2026-06-07)
187241

188242
End-to-end pipeline verified with Kokoro backend:

readme.md

Lines changed: 81 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -7,23 +7,26 @@ CrispTTS is a versatile command-line Text-to-Speech (TTS) tool designed for synt
77
| Project | Role |
88
|---|---|
99
| **[Susurrus](https://github.com/CrispStrobe/Susurrus)** | Python GUI + CLI — 30+ ASR, 12 TTS, translation |
10-
| **[CrispASR](https://github.com/CrispStrobe/CrispASR)** | C++ ASR/TTS engine — 24+ backends, ggml inference |
11-
| **CrispTTS** | This repo — Python TTS CLI with 28+ handlers |
10+
| **[CrispASR](https://github.com/CrispStrobe/CrispASR)** | C++ ASR/TTS engine — 26+ ASR, 14+ TTS backends, ggml inference |
11+
| **CrispTTS** | This repo — Python TTS CLI with 31+ handlers |
1212
| **[CrisperWeaver](https://github.com/CrispStrobe/CrisperWeaver)** | Flutter transcription app — desktop + mobile |
1313

1414
NOTE: This is in experimental / work in progress state. Some Python-only models may be broken due to dependency conflicts. The CrispASR-based handlers (`crispasr_*`) are the most reliable — they use native C++ inference with no Python ML dependencies.
1515

1616
## Features
1717

18-
- **28+ TTS Engine Support**:
19-
- **CrispASR native C++ engines** (7 backends, auto-download, no Python ML deps):
18+
- **31+ TTS Engine Support**:
19+
- **CrispASR native C++ engines** (10 backends, auto-download, no Python ML deps):
2020
- Kokoro (multilingual, Apache 2.0)
2121
- Orpheus + Kartoffel-Orpheus DE (19 German speakers, llama3.2 license)
2222
- Qwen3-TTS (voice cloning + voice design, Apache 2.0)
2323
- Chatterbox (CFM synthesis, MIT)
2424
- VibeVoice TTS (voice cloning)
2525
- IndexTTS (zero-shot cloning, Apache 2.0)
2626
- VoxCPM2 (48 kHz, 30 languages, Apache 2.0)
27+
- F5-TTS (flow-matching, voice cloning, Apache 2.0)
28+
- MeloTTS (VITS2, 44.1 kHz, MIT)
29+
- Piper (250+ community voices, 30+ languages — faster than Python Piper)
2730
- Microsoft Edge TTS (cloud-based, requires `edge-tts`)
2831
- Coqui TTS (XTTS v2, VITS, etc.)
2932
- Piper (local ONNX, requires `piper-tts`)
@@ -39,12 +42,15 @@ NOTE: This is in experimental / work in progress state. Some Python-only models
3942
- TTS.cpp (GGUF models)
4043
- Zonos (acoustic conditioning)
4144
- Chatterbox Python (Kartoffelbox)
42-
- **AI Audio Watermarking & Provenance** (EU AI Act compliant):
43-
- Spread-spectrum watermark (always on, imperceptible, ~-46 dB)
45+
- **AI Audio Watermarking & Provenance** (EU AI Act Art. 50 compliant):
46+
- WavMark neural watermark (MIT license — code + model weights; `pip install wavmark`)
47+
- Spread-spectrum watermark (always on, imperceptible, ~38 dB SNR)
4448
- AudioSeal neural watermark (optional upgrade via `pip install audioseal` or CrispASR GGUF)
45-
- WAV LIST/INFO and MP3 ID3v2 metadata marking audio as AI-generated
49+
- WAV LIST/INFO, MP3 ID3v2, FLAC Vorbis comment, and Opus/OGG metadata marking audio as AI-generated
4650
- C2PA content credentials signing (optional, `pip install c2pa-python`)
47-
- Voice-cloning consent gate (`--i-have-rights`)
51+
- Voice-cloning consent gate (`--i-have-rights` CLI / `"i_have_rights": true` API)
52+
- Spoken AI disclaimer prepended to voice-cloned audio (Art. 50(4))
53+
- Persistent consent audit log at `~/.cache/crisptts/consent_audit.log`
4854
- **CrispASR Integration**:
4955
- `--verify`: ASR roundtrip verification of TTS output quality
5056
- `--translate`: Pre-synthesis translation (EN→DE via m2m100/MadLad)
@@ -59,6 +65,8 @@ NOTE: This is in experimental / work in progress state. Some Python-only models
5965
- **OpenAI-Compatible API Server** (`--server`):
6066
- `POST /v1/audio/speech` — drop-in replacement for OpenAI TTS
6167
- `GET /v1/audio/models` — list all configured models
68+
- Voice-cloning consent gate (returns 403 if `i_have_rights` missing)
69+
- All responses watermarked + metadata-injected
6270
- **Text Input Flexibility**: Synthesize from CLI, `.txt`, `.md`, `.html`, `.pdf`, `.epub`
6371
- **Smart Text Chunking**: Automatic sentence-boundary splitting for long texts
6472
- **Customizable Output**: Save audio to `.wav`, `.mp3`, `.flac`, or `.opus`
@@ -68,7 +76,7 @@ NOTE: This is in experimental / work in progress state. Some Python-only models
6876
- **Comprehensive Testing**:
6977
- `--test-all`: Test all models with default voices
7078
- `--test-all-speakers`: Test all models with all configured voices
71-
- 160+ unit and live tests
79+
- 212 unit and live tests
7280
- **Modular Design**: `config.py` + `utils.py` + `handlers/` + `main.py`
7381
- **Logging**: Configurable logging levels
7482
- **Automatic Patching**: Runtime monkeypatches for library compatibility
@@ -86,6 +94,7 @@ crisptts_project/
8694
├── decoder.py # User-provided decoder for Orpheus models (if used)
8795
├── handlers/ # Package for individual TTS engine handlers
8896
│ ├── __init__.py # Makes 'handlers' a package, exports handler functions
97+
│ ├── crispasr_handler.py # CrispASR native C++ TTS (10 backends)
8998
│ ├── edge_handler.py # Edge TTS cloud service handler
9099
│ ├── piper_handler.py # Piper TTS (ONNX) handler
91100
│ ├── orpheus_gguf_handler.py # Local Orpheus GGUF model handler
@@ -98,16 +107,21 @@ crisptts_project/
98107
│ ├── kokoro_onnx_handler.py # Kokoro (multilingual but no German) ONNX handler
99108
│ ├── llasa_hybrid_handler.py # LLaSA Hybrid handler
100109
│ ├── tts_cpp_handler.py # TTS.cpp handler supporting GGUF models
110+
│ ├── f5_tts_handler.py # F5-TTS handler (MLX/PyTorch)
111+
│ ├── zonos_handler.py # Zonos acoustic conditioning handler
112+
│ ├── chatterbox_handler.py # Chatterbox/Kartoffelbox handler
101113
│ └── mlx_audio_handler.py # Handler for mlx-audio library (e.g., Bark)
114+
├── tests/ # Unit and integration tests
102115
├── requirements.txt # Python package dependencies
116+
├── pyproject.toml # Project metadata and build config
103117
└── README.md # This documentation file
104118
```
105119

106120
## Setup and Installation
107121

108122
### Prerequisites
109123

110-
- Python and `pip` for installing packages
124+
- Python 3.10+ and `pip` for installing packages
111125
- For `mlx-audio` based models: Apple Silicon Mac is required for GPU acceleration
112126
- For `TTS.cpp` a C++ compiler and CMake are required to build the engine
113127

@@ -125,11 +139,18 @@ crisptts_project/
125139
```
126140

127141
3. **Install Dependencies**:
128-
A `requirements.txt` file is provided. Install the necessary packages:
129142
```bash
130143
pip install -r requirements.txt
131144
```
132145

146+
Optional feature groups:
147+
```bash
148+
pip install crisptts[watermark-mit] # WavMark neural watermark (MIT license)
149+
pip install crisptts[metadata] # FLAC/Opus metadata via mutagen
150+
pip install crisptts[provenance] # C2PA content credentials
151+
pip install crisptts[dev] # ruff, bandit, pytest
152+
```
153+
133154
> **Note**: Some libraries like PyTorch, NeMo, LlamaCPP, and `mlx-audio` can have specific installation needs depending on your OS and hardware (e.g., CUDA for Nvidia GPUs, Metal for Apple Silicon). Please refer to their official documentation if you encounter issues.
134155
> Ensure you have `ffmpeg` installed and available in your system's PATH if you encounter issues with audio file format conversions or direct playback (some underlying libraries might need it).
135156
@@ -276,6 +297,19 @@ python main.py --server --server-port 8880
276297
# --output speech.wav
277298
```
278299
300+
**Voice cloning (with consent attestation):**
301+
```bash
302+
# CLI
303+
python main.py --model-id coqui_xtts_v2_de_clone --i-have-rights \
304+
--input-text "Hallo" --german-voice-id ref_voice.wav --output-file cloned.wav
305+
306+
# API (include i_have_rights in request body)
307+
curl -X POST http://localhost:8880/v1/audio/speech \
308+
-H "Content-Type: application/json" \
309+
-d '{"model":"crispasr_f5_tts","input":"Hallo","voice":"ref.wav","i_have_rights":true}' \
310+
--output cloned.wav
311+
```
312+
279313
**Change Logging Level (for debugging):**
280314
```bash
281315
python main.py --model-id edge --input-text "Debug Test." --loglevel DEBUG
@@ -291,6 +325,7 @@ python main.py --model-id orpheus_ollama --input-text "Hallo Ollama" --ollama-ap
291325
292326
Refer to the output of `python main.py --list-models` for the currently configured models and their notes. The script supports integration with:
293327
328+
- CrispASR native C++ (10 backends: Kokoro, Orpheus, Qwen3-TTS, Chatterbox, VibeVoice, IndexTTS, VoxCPM2, F5-TTS, MeloTTS, Piper)
294329
- Microsoft Edge TTS
295330
- Piper TTS
296331
- Orpheus GGUF (via llama-cpp-python)
@@ -303,6 +338,9 @@ Refer to the output of `python main.py --list-models` for the currently configur
303338
- Orpheus "Kartoffel" (Transformers-based)
304339
- LLaSA Hybrid (Experimental MLX + PyTorch)
305340
- mlx-audio (e.g., Bark for Apple Silicon)
341+
- F5-TTS (MLX/PyTorch)
342+
- Zonos (acoustic conditioning)
343+
- Chatterbox/Kartoffelbox (Python)
306344
307345
## Adding New TTS Handlers
308346
@@ -359,32 +397,48 @@ CrispASR/CrisperWeaver native runtime), see
359397

360398
## Audio Watermarking & Provenance (EU AI Act Art. 50)
361399

362-
CrispTTS automatically marks all synthesized audio as AI-generated using a multi-layered provenance system ported from [CrispASR](https://github.com/CrispStrobe/CrispASR). Article 50 transparency obligations take effect **2 August 2026**.
400+
CrispTTS automatically marks all synthesized audio as AI-generated using a multi-layered provenance system. Article 50 transparency obligations take effect **2 August 2026**.
401+
402+
All outputs are watermarked — CLI, `--test-all`, and API server responses. CrispASR C++ backends watermark at the binary level; all other handlers are watermarked in Python post-synthesis.
363403

364404
### Layers
365405

366406
| Layer | What | Status | Install |
367407
|-------|------|--------|---------|
408+
| **WavMark** | Neural watermark (MIT license, 16-bit payload, >38 dB SNR) | Auto-detected (preferred) | `pip install wavmark` |
368409
| **Spread-spectrum** | Frequency-domain watermark (32 bins, alpha=0.08, ~38 dB SNR) | Always active | Built-in (numpy) |
369410
| **AudioSeal** | Neural watermark (Meta, 16-bit message, sample-rate aware) | Auto-detected | `pip install audioseal` |
370-
| **WAV/MP3 metadata** | LIST/INFO + ID3v2 TXXX tags | Always active | Built-in |
411+
| **WAV/MP3/FLAC/Opus metadata** | LIST/INFO, ID3v2, Vorbis comments — `AI_GENERATED=true` | Always active | Built-in (FLAC/Opus: `pip install mutagen`) |
371412
| **C2PA credentials** | Signed provenance manifests (`trainedAlgorithmicMedia`) | Opt-in | `pip install c2pa-python` |
372413
| **Spoken disclaimer** | AI disclosure prepended to voice-cloned audio | Auto for cloning | Built-in |
373-
| **Consent gate** | Voice-cloning attestation + audit logging | Required for cloning | Built-in |
374-
| **Post-embed verification** | Watermark detection after file write | Always active | Built-in |
414+
| **Consent gate** | Voice-cloning attestation + persistent audit logging | Required for cloning | Built-in |
415+
416+
**Watermark backend priority**: WavMark (MIT) > AudioSeal (Python) > CrispASR GGUF > spread-spectrum (always-on fallback).
417+
418+
### Voice cloning safety
419+
420+
Voice-cloning models require explicit consent attestation before synthesis is allowed:
421+
422+
- **CLI**: `--i-have-rights` flag required (synthesis blocked without it)
423+
- **API**: `"i_have_rights": true` in request body (returns 403 without it)
424+
- **Detection**: triggered by handler key, model ID keywords (`clone`, `xtts`, `zeroshot`, `vibevoice`, `indextts`, `voxcpm2`, `qwen3_tts`, `f5_tts`, `zonos`, `chatterbox`), or `.wav` voice path
425+
- **Audit log**: written to stderr AND `~/.cache/crisptts/consent_audit.log`
426+
- **Spoken disclaimer**: "This audio was generated by artificial intelligence." prepended to cloned output (generated via CrispASR kokoro, Edge TTS fallback, beep marker last resort)
375427

376428
### Compliance comparison across the Crisp ecosystem
377429

378430
| Feature | CrispTTS | CrispASR | CrisperWeaver |
379431
|---------|----------|----------|---------------|
380432
| Spread-spectrum watermark | numpy (Python) | C++ header-only | Dart LSB + native FFI |
433+
| WavMark neural watermark (MIT) | Python (wavmark) |||
381434
| AudioSeal neural watermark | Python + crispasr GGUF | C++ ggml (GGUF) | via CrispASR FFI |
382435
| WAV LIST/INFO metadata | ISFT + ICMT | ISFT + ICMT | ISFT + ICMT + IART + ICRD |
383436
| MP3 ID3v2 tags | TXXX (AI_GENERATED) | TXXX (AI_GENERATED) | TXXX (AI_GENERATED) |
437+
| FLAC/Opus metadata | Vorbis comments (mutagen) |||
384438
| C2PA content credentials | c2pa-python (optional) | c2pa-c (compile-time) ||
385-
| Spoken AI disclaimer | Edge TTS / beep fallback | Native TTS (cached) | Beep marker |
386-
| Voice-cloning consent gate | `--i-have-rights` CLI | `--i-have-rights` CLI + server JSON | GDPR Art. 9(2)(a) consent files |
387-
| Consent audit logging | `[CONSENT]` stderr | `[CONSENT]` stderr | `[CONSENT]` log + `.consent.json` |
439+
| Spoken AI disclaimer | CrispASR kokoro / Edge TTS / beep | Native TTS (cached) | Beep marker |
440+
| Voice-cloning consent gate | CLI + API (403) | CLI + server JSON | GDPR Art. 9(2)(a) consent files |
441+
| Consent audit logging | stderr + `consent_audit.log` | `[CONSENT]` stderr | `[CONSENT]` log + `.consent.json` |
388442
| Post-embed verification | detect after save | detect after save | detect after embed |
389443
| Watermark detection CLI | `--detect-watermark` | `--detect-watermark` | detect in service |
390444
| Cross-project detection | Yes (shared PRNG key) | Yes (shared PRNG key) | Yes (via CrispASR FFI) |
@@ -395,16 +449,16 @@ CrispTTS automatically marks all synthesized audio as AI-generated using a multi
395449
# Default: spread-spectrum watermark + metadata (no extra deps)
396450
python main.py --model-id edge --input-text "Hallo" --output-file out.mp3
397451
398-
# With AudioSeal neural watermark (auto-detected if installed)
399-
pip install audioseal
452+
# With WavMark neural watermark (MIT, preferred)
453+
pip install wavmark
400454
python main.py --model-id edge --input-text "Hallo" --output-file out.mp3
401455
402456
# With C2PA content credentials
403457
pip install c2pa-python
404458
python main.py --c2pa-cert cert.pem --c2pa-key key.pem --model-id edge --input-text "Hallo" --output-file out.mp3
405459
406460
# Voice-cloning models require consent attestation (spoken disclaimer auto-prepended)
407-
python main.py --model-id coqui_xtts_v2 --i-have-rights --input-text "Hallo" --output-file out.wav
461+
python main.py --model-id coqui_xtts_v2_de_clone --i-have-rights --input-text "Hallo" --output-file out.wav
408462
409463
# Detect watermark in existing audio
410464
python main.py --detect-watermark out.wav
@@ -456,11 +510,14 @@ python server.py --host 0.0.0.0 --port 8880
456510
"input": "Hallo, wie geht es Ihnen?",
457511
"voice": "af_heart",
458512
"response_format": "wav",
459-
"speed": 1.0
513+
"speed": 1.0,
514+
"i_have_rights": false
460515
}
461516
```
462517

463-
Response: audio bytes with appropriate Content-Type header. All output is automatically watermarked.
518+
The `i_have_rights` field is required (and must be `true`) for voice-cloning models. Omit it or set to `false` for non-cloning models.
519+
520+
Response: audio bytes with appropriate Content-Type header. All output is automatically watermarked. Response includes `X-CrispTTS-Watermarked: true` header.
464521

465522
## Troubleshooting & Notes
466523

@@ -492,4 +549,4 @@ python main.py --model-id crispasr_kokoro \
492549
493550
**RAM Usage**: Local GGUF and large Transformer models can be memory-intensive. Ensure your system has sufficient RAM.
494551
495-
**Paths**: When providing paths for `--input-file`, `--output-file`, or speaker WAV files (`--german-voice-id`), use appropriate relative or absolute paths.
552+
**Paths**: When providing paths for `--input-file`, `--output-file`, or speaker WAV files (`--german-voice-id`), use appropriate relative or absolute paths.

0 commit comments

Comments
 (0)