VAE-based Phoneme Alignment Using Gradient Annealing and SSL Acoustic Features

Accepted to INTERSPEECH 2024 arXiv preprint

Training and inference code is available at https://github.com/CyberAgentAILab/vae_speech_align

Alignment examples

• annotated: Manually annotated phoneme boundaries in the corpus
• Proposed: Predicted boundaries using proposed method
• MFA: Predicted boundaries using Montreal Forced Aligner
• CTC: Predicted boundaries using CTC forced alignment
• OTA: Predicted boundaries using "One TTS alignment to rule them all"

CSJ dataset

TIMIT dataset

Buckeye dataset