A New Technology You Should Know: Fish-Speech

In this article, we’ll explore what makes Fish Speech a game-changer in the world of TTS technology.

What is Fish Speech?

Fish Speech, now OpenAudio, aims to provide state-of-the-art text-to-speech solutions that are both powerful and accessible. The project has redefined TTS by introducing models capable of generating natural-sounding speech from text input, supporting multiple languages and a wide range of emotional tones.

The first model in this series is OpenAudio-S1, which builds upon the foundation set by its predecessor, Fish-Speech. OpenAudio-S1 comes in two versions: OpenAudio-S1 and OpenAudio-S1-mini. While both models are designed for high-quality speech synthesis, they cater to different needs—S1 offers a more comprehensive feature set, while S1-mini is a distilled version with core capabilities ideal for basic use cases.

Key Features of OpenAudio

1. Exceptional TTS Quality

OpenAudio-S1 has achieved top rankings on TTS-Arena2, a benchmark platform for evaluating text-to-speech systems. With a Word Error Rate (WER) of 0.008 and Character Error Rate (CER) of 0.004, OpenAudio-S1 delivers superior accuracy in speech generation.

2. Multilingual Support

OpenAudio supports a wide range of languages, including English, Japanese, Korean, Chinese, French, German, Arabic, and Spanish. This means you can generate speech in multiple languages with just a few clicks—no complex setup required.

3. Advanced Emotional and Tone Control

With OpenAudio-S1, you can fine-tune the emotional tone of the generated speech. From basic emotions like "happy" or "angry" to more nuanced tones such as "sincere," "sarcastic," or "confident," the model offers a rich palette of options to bring your text to life.

Here’s a sneak peek at some of the supported emotional and tone markers:

• 
  Basic Emotions: Happy, sad, excited, surprised, satisfied, delighted, scared, worried, upset, nervous, frustrated, depressed, empathetic, embarrassed, disgusted, moved, proud, relaxed, grateful, confident, interested, curious, confused, joyful.
  


• 
  Advanced Tones: Sarcasm, irony, enthusiasm, calmness, urgency, happiness, sadness, anger, fear, excitement.
  

4. No Phoneme Dependency

Unlike traditional TTS systems that rely on phonemes or syllables, OpenAudio-S1 operates at the character level. This allows it to handle any script without prior knowledge of the language’s sound system, making it ideal for multilingual and less common scripts.

5. Fast and Deployable

OpenAudio-S1 is optimized for speed, with torch compile reducing inference time by a factor of 7 on an Nvidia RTX 4090 GPU. Plus, it comes with built-in support for web interfaces (via Gradio) and GUIs (using PyQt6), making it easy to integrate into existing workflows.

How to Get Started

For Developers

If you’re a developer looking to integrate OpenAudio into your project, you’ll appreciate its ease of use and flexibility. The model supports both zero-shot and few-shot TTS, meaning you can generate high-quality speech with minimal or no prior examples.

For detailed installation guides and best practices for voice cloning, check out the official documentation at