StyleTTS2 Hindi + Hinglish: Monica
A bilingual Hindi, English and code-mixed Hinglish voice ("Monica") built on StyleTTS2 by superU AI for low-latency conversational voice agents in India.
It reads Devanagari Hindi, English, and sentences that switch between the two mid-sentence, such as "आपका order कल deliver हो जाएगा", with no separate language-identification model.
This is the model described in our paper "Extending StyleTTS2 for Hindi and Bilingual Speech Synthesis: A Word-Level G2P Framework" (superU AI, April 2026).
English-only voice: see superU-AI/style-tts-2-english-anika.
Results
Evaluated on 150 held-out sentences (50 English, 50 Hindi, 50 Hinglish), single NVIDIA A100.
Naturalness is predicted MOS from UTMOS (utmos22_strong), not a human listening study.
WER uses Whisper large-v3 for English and Hindi, and
Shunya Labs zero-stt-hinglish for Hinglish,
since monolingual ASR mistranscribes code-switched speech.
| Condition | System | UTMOS ↑ | WER (%) ↓ |
|---|---|---|---|
| English | This model | 3.785 | 2.66 |
| English | StyleTTS2 LibriTTS (unmodified) | 4.446 | 0.68 |
| Hindi | This model | 3.003 | 11.77 |
| Hindi | Indic Parler-TTS | 3.116 | 31.28 |
| Hinglish | This model | 3.328 | 10.32 |
| Hinglish | Svara-TTS | 4.087 | 13.23 |
Latency: mean real-time factor of 0.045 on an A100 (unmodified English StyleTTS2: 0.0385), so the bilingual pipeline adds about 1.17x overhead and stays far below real time.
The trade-off: larger autoregressive and Hindi-only models score higher on predicted naturalness, while this model is markedly more intelligible on Hindi and Hinglish and much faster, which is what matters for a voice agent that has to be understood on the first try.
Ablation: word-level vs sentence-level G2P
Same checkpoint and weights; only the text front-end changes.
| Condition | Sentence-level WER (%) | Word-level WER (%) |
|---|---|---|
| English | 2.91 | 2.66 |
| Hindi | 14.38 | 11.77 |
| Hinglish | 28.91 | 10.32 |
Routing a whole mixed sentence through one phonemizer destroys the English words in it. Phonemizing per word is what makes Hinglish work.
How it works
StyleTTS2 is English-only out of the box. Four changes extend it:
- Word-level G2P routing. The text front-end tokenizes into words and phonemizes each one
separately. A word containing any Devanagari character (U+0900–U+097F) goes to
aksharamukha for Hindi IPA; anything else goes to
eng_to_ipa. The script check is deterministic and zero-latency, so no language-ID model is needed. - Custom pronunciation dictionary. A case-insensitive lookup in front of G2P overrides pronunciations for proper nouns, loanwords, brand names and symbols such as ₹.
- Expanded phoneme vocabulary. Hindi IPA symbols (retroflex ʈ ɖ ɳ ɽ, aspiration ʰ, nasalization and length marks, and Hindi vowels) are appended after the original English symbols, so the pretrained English embeddings keep their indices. Without them, Hindi phonemes map to the padding token and are silently erased.
- Multilingual PL-BERT (papercup-ai/multilingual-pl-bert) replaces the English-only PL-BERT, for coherent prosody and duration on Hindi input.
The model then sees an ordinary flat phoneme sequence; the acoustic architecture is unchanged.
Model details
| Architecture | StyleTTS2 (multispeaker config) |
| Decoder | HiFi-GAN |
| Base checkpoint | StyleTTS2 LibriTTS 2nd-stage |
| PL-BERT | Multilingual PL-BERT |
| SLM discriminator | microsoft/wavlm-base-plus |
| Sample rate | 24 kHz |
| Style dim | 128 |
Training
Data: 24 hours of synthetic speech: 10 h Hindi (Devanagari text, covering retroflex-heavy words and conjunct consonants), 10 h English (to prevent catastrophic forgetting) and 4 h code-mixed Hinglish, all at 24 kHz, split 80/20 train/validation.
| Parameter | Value |
|---|---|
| GPU | 1x NVIDIA A100 80GB, about 2 days |
| Epochs | 90 (2nd stage only; stopped on validation-loss convergence) |
| Batch size | 2 |
| Max frame length | 300 |
| Learning rate | 1e-4 (PL-BERT 1e-5, acoustic modules 1e-5) |
| Style diffusion starts | epoch 10 |
| Joint / SLM adversarial training starts | epoch 15 |
| Final validation loss | ~0.31 (duration 0.41) |
No component was frozen; all modules were fine-tuned jointly.
Files
Hindi-MONIKA/
config_ft_hindi.yml # fine-tuning config, needed for inference
epoch_2nd_000XX.pth # checkpoints every 5 epochs; epoch_2nd_00089.pth is the final one
train.log # full training log
Usage
The weights load with the official StyleTTS2 inference code, using config_ft_hindi.yml and
epoch_2nd_00089.pth, with Utils/PLBERT_Multilingual/ populated from
papercup-ai/multilingual-pl-bert.
git clone https://github.com/yl4579/StyleTTS2.git
git xet install # weights are stored with Git Xet: https://hf.co/docs/hub/git-xet
git clone https://huggingface.co/superU-AI/Style-tts-hindi-monica
Important: Hindi and Hinglish input must go through the word-level text front-end and the
expanded symbol set described above. The stock StyleTTS2 TextCleaner will drop Hindi phonemes.
The front-end code is not included in this repository yet.
Limitations
- Synthetic training data. All training audio was generated, not recorded by native speakers, and this is the main ceiling on naturalness, especially for Hindi.
- Romanized Hindi is read as English. "chai" or "roti" in Latin script get English phonemes and a slight English accent. Write Hindi in Devanagari.
- Automated evaluation only. UTMOS and ASR-based WER are proxies; no human MOS study yet.
- English quality trade-off. English UTMOS drops from 4.446 to 3.785 versus the unmodified model, though English WER stays low at 2.66%.
Citation
@techreport{superu2026styletts2hindi,
title = {Extending StyleTTS2 for Hindi and Bilingual Speech Synthesis: A Word-Level G2P Framework},
author = {{superU AI}},
institution = {superU AI},
year = {2026},
month = {April}
}
Built on StyleTTS2 (Li et al., NeurIPS 2023).