tiny-voca Kiswahili (55k)
A 4.54M-parameter Swahili voice small enough to run offline on a low-end Android phone. 21.4 MB of ONNX, CPU only, no cloud: on a Redmi 9A (ARMv7, Android 11) it synthesises a sentence at RTF 0.16-0.21 in 71-75 MB of RAM.
It is trained from scratch with the tiny-voca recipe (Apache-2.0) by Nazeem (@n4ze3m), not fine-tuned from the English checkpoint. The same export is the voice of an offline reader for the Kenyan constitution, which highlights each spoken word from the model's own phoneme durations.
Sample (rendered by run_onnx.py from this exact ONNX file, voice index 0):
sample-0.wavβ "Habari za asubuhi! Karibu Kenya." (4.96 s)sample-1.wavβ "Leo ni Jumatatu, tarehe kumi na tisa Novemba." (5.77 s)sample-2.wavβ "Bei ya unga ni shilingi elfu moja na mia tano." (5.70 s)
Model details
| Architecture | tiny-voca: source-filter vocoder with a duration head, 4 encoder layers |
| Parameters | 4.54M |
| Size | 21.4 MB ONNX (fp32); 18 MB PyTorch checkpoint |
| Sample rate | 24 kHz, hop 256 samples (10.67 ms per frame) |
| Input | phoneme ids from a 76-symbol inventory, one stress mark per token |
| Voices | 7 speaker ids (1-6, 8), all from WAXAL's Swahili speakers |
| Limit | 512 tokens per call; 3,000 frames (32 s) of audio |
| Training steps | 55,000 of a planned 104,000 |
Training
- Data:
swa_ttsfrom google/WaxalNLP (CC-BY-SA-4.0), 24 kHz resampled. 2,497 utterances (~13 h) from 7 speakers passed forced alignment (facebook/wav2vec2-lv-60-espeak-cv-ft, 55% gate); those are the training set. Splits hold out sentence text, not files - WAXAL reuses prompt sentences across recordings. - Text front end: a deterministic Swahili letter-to-phone transducer
(
chβtΚ,shβΚ,nyβn j,ng'βΕ,th/dhβΞΈ/Γ°, penultimate-syllable stress), so no grapheme model or attention is involved. Digits, money, dates and legal citations must be expanded to words before phonemisation; the model itself only ever sees phonemes. - Hardware: Kaggle free tier, T4 x2. Stopped at 55k steps when the stage-8 samples were clear to listen to, rather than spending the remaining GPU hours on a step count.
Interface
Inputs (dynamic length on axis 1):
| name | shape | dtype | meaning |
|---|---|---|---|
tokens |
[1, L] | int64 | phoneme ids, symbols.json |
stress |
[1, L] | int64 | 0 none, 1 primary, 2 secondary |
voice |
[1] | int64 | speaker index 0-6 |
speed |
[1] | float32 | duration divisor; 1.2 = 20% faster |
pitch_shift |
[1] | float32 | log-Hz offset, 0 = unchanged |
pitch_range |
[1] | float32 | prosody spread multiplier, 1 = unchanged |
Outputs:
| name | shape | meaning |
|---|---|---|
audio |
[1, N] | float32 mono at 24 kHz, peak below 0.5 |
durations |
[1, L] | frames per token; sum(durations) * 256 == N |
durations is the alignment: chunk i covers samples
[sum(durations[:i]) * 256, sum(durations[:i+1]) * 256). That is exact, not an
estimate, which is why word-level highlighting can come from the same forward
pass as the audio.
Usage
pip install onnxruntime numpy
python run_onnx.py 0 out.wav
cases.json contains three sentences with pre-computed tokens and stress, so
the demo runs without the front end. For arbitrary text, use tiny-voca's
Swahili front end (TINY_VOCA_FRONTEND=sw) plus a Swahili verbalizer for
digits, dates, money and legal references, then feed the ids and stress arrays.
import numpy as np, onnxruntime as ort
sess = ort.InferenceSession("swahili.onnx", providers=["CPUExecutionProvider"])
audio, durations = sess.run(None, {
"tokens": np.array([tokens], dtype=np.int64),
"stress": np.array([stress], dtype=np.int64),
"voice": np.array([0], dtype=np.int64),
"speed": np.array([1.0], dtype=np.float32),
"pitch_shift": np.array([0.0], dtype=np.float32),
"pitch_range": np.array([1.0], dtype=np.float32),
})
Measured on a Redmi 9A (220233L2G, Android 11, armeabi-v7a)
ONNX Runtime 1.25.0, 4 threads, warm runs:
| sentence | audio | synthesis | RTF |
|---|---|---|---|
| Habari za asubuhi! Karibu Kenya. | 4.96 s | 798 ms | 0.161 |
| Leo ni Jumatatu, tarehe kumi na tisa Novemba. | 5.77 s | 1188 ms | 0.206 |
| Bei ya unga ni shilingi elfu moja na mia tano. | 5.70 s | 1167 ms | 0.205 |
Session creation 5.3 s, PSS 71-75 MB while synthesising. The same export reproduces the PyTorch duration counts and audio length sample for sample; waveforms correlate 0.94-0.98 with the desktop renderer, and two desktop runs of the same input differ by as much.
Known limitations
- Prototype voice. No native-speaker listening study and no WER/MOS evaluation yet. Expect uneven prosody on long sentences.
- Each utterance carries a breathy tail (~1.9 s after the last word when synthesised sentence by sentence). Trim to the last word plus 0.25 s; without trimming the reading rate drops to about 70 wpm.
- Numerals and abbreviations are the caller's job. "2022" must arrive as "elfu mbili na ishirini na mbili"; the model cannot read digits.
- Read speech of seven speakers. No claim of dialect coverage, code switching, or emotional style. English is not supported.
- Not a cloning model. There is no speaker encoder; the seven ids are fixed.
Licence and provenance
- Weights: CC-BY-SA-4.0, inherited from WAXAL's share-alike licence. If you build on these weights, attribute the dataset and share your derivatives alike. The included PyTorch checkpoint is under the same terms.
- Training data: google/WaxalNLP
(
swa_tts, CC-BY-SA-4.0). No private or personal recordings were used, and no cloned voice is included; every voice here is a WAXAL speaker. - Code: tiny-voca by Nazeem, Apache-2.0 (https://github.com/n4ze3m/tiny-voca).
- The Swahili front end, verbalizer and on-device app were built for this release and are not part of tiny-voca upstream.
Files
| file | what |
|---|---|
swahili.onnx |
release graph, internal noise, 21.4 MB |
swahili-inference.pt |
PyTorch checkpoint (step 55,000) for fine-tuning |
metadata.json |
rate, hop, voices, limits, frontend identity |
recipe.json |
training recipe and data hashes |
symbols.json |
the 76 phoneme ids, index = id |
config.json |
upstream config (voices, vocab size) |
cases.json |
three demo sentences with token ids and stress |
run_onnx.py |
renders a case to 24 kHz wav |
sample-0/1/2.wav |
the three cases, rendered by this ONNX |
- Downloads last month
- 20