Update README.md
Browse files
README.md
CHANGED
@@ -1,11 +1,94 @@
|
|
1 |
-
---
|
2 |
-
license: mit
|
3 |
-
language:
|
4 |
-
- en
|
5 |
-
tags:
|
6 |
-
- speech
|
7 |
-
-
|
8 |
-
-
|
9 |
-
|
10 |
-
|
11 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
tags:
|
6 |
+
- text-to-speech
|
7 |
+
- speech generation
|
8 |
+
- voice-cloning
|
9 |
+
pipeline_tag: text-to-speech
|
10 |
+
---
|
11 |
+
|
12 |
+
<img width="800" alt="cb-big2" src="https://github.com/user-attachments/assets/bd8c5f03-e91d-4ee5-b680-57355da204d1" />
|
13 |
+
|
14 |
+
<h1 style="font-size: 32px">Chatterbox TTS</h1>
|
15 |
+
|
16 |
+
<div style="display: flex; align-items: center; gap: 12px">
|
17 |
+
<a href="https://resemble-ai.github.io/chatterbox_demopage/">
|
18 |
+
<img src="https://img.shields.io/badge/listen-demo_samples-blue" alt="Listen to Demo Samples" />
|
19 |
+
</a>
|
20 |
+
<a href="https://huggingface.co/spaces/ResembleAI/Chatterbox">
|
21 |
+
<img src="https://huggingface.co/datasets/huggingface/badges/resolve/main/open-in-hf-spaces-sm.svg" alt="Open in HF Spaces" />
|
22 |
+
</a>
|
23 |
+
<a href="https://podonos.com/resembleai/chatterbox">
|
24 |
+
<img src="https://static-public.podonos.com/badges/insight-on-pdns-sm-dark.svg" alt="Insight on Podos" />
|
25 |
+
</a>
|
26 |
+
</div>
|
27 |
+
|
28 |
+
<div style="display: flex; align-items: center; gap: 8px;">
|
29 |
+
<span style="font-style: italic;white-space: pre-wrap">Made with ❤️ by</span>
|
30 |
+
<img width="100" alt="resemble-logo-horizontal" src="https://github.com/user-attachments/assets/35cf756b-3506-4943-9c72-c05ddfa4e525" />
|
31 |
+
</div>
|
32 |
+
|
33 |
+
|
34 |
+
We're excited to introduce Chatterbox, [Resemble AI's](https://resemble.ai) first production-grade open source TTS model. Licensed under MIT, Chatterbox has been benchmarked against leading closed-source systems like ElevenLabs, and is consistently preferred in side-by-side evaluations.
|
35 |
+
|
36 |
+
Whether you're working on memes, videos, games, or AI agents, Chatterbox brings your content to life. It's also the first open source TTS model to support **emotion exaggeration control**, a powerful feature that makes your voices stand out. Try it now on our [Hugging Face Gradio app.](https://huggingface.co/spaces/ResembleAI/Chatterbox)
|
37 |
+
|
38 |
+
If you like the model but need to scale or tune it for higher accuracy, check out our competitively priced TTS service (<a href="https://resemble.ai">link</a>). It delivers reliable performance with ultra-low latency of sub 200ms—ideal for production use in agents, applications, or interactive media.
|
39 |
+
|
40 |
+
# Key Details
|
41 |
+
- SoTA zeroshot TTS
|
42 |
+
- 0.5B Llama backbone
|
43 |
+
- Unique exaggeration/intensity control
|
44 |
+
- Ultra-stable with alignment-informed inference
|
45 |
+
- Trained on 0.5M hours of cleaned data
|
46 |
+
- Watermarked outputs
|
47 |
+
- Easy voice conversion script
|
48 |
+
- [Outperforms ElevenLabs](https://podonos.com/resembleai/chatterbox)
|
49 |
+
|
50 |
+
# Tips
|
51 |
+
- **General Use (TTS and Voice Agents):**
|
52 |
+
- The default settings (`exaggeration=0.5`, `cfg=0.5`) work well for most prompts.
|
53 |
+
- If the reference speaker has a fast speaking style, lowering `cfg` to around `0.3` can improve pacing.
|
54 |
+
|
55 |
+
- **Expressive or Dramatic Speech:**
|
56 |
+
- Try lower `cfg` values (e.g. `~0.3`) and increase `exaggeration` to around `0.7` or higher.
|
57 |
+
- Higher `exaggeration` tends to speed up speech; reducing `cfg` helps compensate with slower, more deliberate pacing.
|
58 |
+
|
59 |
+
|
60 |
+
# Installation
|
61 |
+
```
|
62 |
+
pip install chatterbox-tts
|
63 |
+
```
|
64 |
+
|
65 |
+
|
66 |
+
# Usage
|
67 |
+
```python
|
68 |
+
import torchaudio as ta
|
69 |
+
from chatterbox.tts import ChatterboxTTS
|
70 |
+
|
71 |
+
model = ChatterboxTTS.from_pretrained(device="cuda")
|
72 |
+
|
73 |
+
text = "Ezreal and Jinx teamed up with Ahri, Yasuo, and Teemo to take down the enemy's Nexus in an epic late-game pentakill."
|
74 |
+
wav = model.generate(text)
|
75 |
+
ta.save("test-1.wav", wav, model.sr)
|
76 |
+
|
77 |
+
# If you want to synthesize with a different voice, specify the audio prompt
|
78 |
+
AUDIO_PROMPT_PATH="YOUR_FILE.wav"
|
79 |
+
wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH)
|
80 |
+
ta.save("test-2.wav", wav, model.sr)
|
81 |
+
```
|
82 |
+
See `example_tts.py` for more examples.
|
83 |
+
|
84 |
+
# Acknowledgements
|
85 |
+
- [Cosyvoice](https://github.com/FunAudioLLM/CosyVoice)
|
86 |
+
- [HiFT-GAN](https://github.com/yl4579/HiFTNet)
|
87 |
+
- [Llama 3](https://github.com/meta-llama/llama3)
|
88 |
+
|
89 |
+
# Built-in PerTh Watermarking for Responsible AI
|
90 |
+
|
91 |
+
Every audio file generated by Chatterbox includes [Resemble AI's Perth (Perceptual Threshold) Watermarker](https://github.com/resemble-ai/perth) - imperceptible neural watermarks that survive MP3 compression, audio editing, and common manipulations while maintaining nearly 100% detection accuracy.
|
92 |
+
|
93 |
+
# Disclaimer
|
94 |
+
Don't use this model to do bad things. Prompts are sourced from freely available data on the internet.
|