Open-Source FastSpeech2 TTS Built from Scratch
π‘See how PostNet, pause gating, and discriminator initialization changed a from-scratch TTS systemβs quality.
β‘ 30-Second TL;DR
What Changed
The pipeline combines CTC forced alignment, FastSpeech2, a standalone PostNet, and fine-tuned HiFi-GAN.
Why It Matters
The project offers a practical reference for developers who want to understand and reproduce modern TTS components without relying on opaque end-to-end systems. Its findings also highlight that vocoder discriminator initialization and duration modeling can materially affect perceived quality.
What To Do Next
Clone the Ash-TTS repository and reproduce the PostNet ablation and HiFi-GAN discriminator initialization on a small LJSpeech subset before adapting the pipeline to your own corpus.
Key Points
- β’The pipeline combines CTC forced alignment, FastSpeech2, a standalone PostNet, and fine-tuned HiFi-GAN.
- β’PostNet improved WER from 14.5% to 8.8%, CER from 8.2% to 5.1%, and MCD from 7.54 dB to 7.02 dB.
- β’Natural pauses are created by inserting <sil> tokens when alignment gaps last at least 6 frames and have sufficiently low energy.
- β’HiFi-GAN training quality depended on initializing discriminators from a universal checkpoint rather than randomly.
- β’The phoneme-based model handles unseen words but struggles with unfamiliar phoneme sequence patterns such as 'higgledy-piggledy'.
Weekly AI Recap
Read this week's curated digest of top AI events β
πRelated Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/MachineLearning β
This is a summary, not the original. Read the source, or get the weekly briefing.
Weekly AI briefing
One email a week. Unsubscribe anytime.