tiny-voca Kiswahili (55k)

A 4.54M-parameter Swahili voice small enough to run offline on a low-end Android phone. 21.4 MB of ONNX, CPU only, no cloud: on a Redmi 9A (ARMv7, Android 11) it synthesises a sentence at RTF 0.16-0.21 in 71-75 MB of RAM.

It is trained from scratch with the tiny-voca recipe (Apache-2.0) by Nazeem (@n4ze3m), not fine-tuned from the English checkpoint. The same export is the voice of an offline reader for the Kenyan constitution, which highlights each spoken word from the model's own phoneme durations.

Sample (rendered by run_onnx.py from this exact ONNX file, voice index 0):

  • sample-0.wav β€” "Habari za asubuhi! Karibu Kenya." (4.96 s)
  • sample-1.wav β€” "Leo ni Jumatatu, tarehe kumi na tisa Novemba." (5.77 s)
  • sample-2.wav β€” "Bei ya unga ni shilingi elfu moja na mia tano." (5.70 s)

Model details

Architecture tiny-voca: source-filter vocoder with a duration head, 4 encoder layers
Parameters 4.54M
Size 21.4 MB ONNX (fp32); 18 MB PyTorch checkpoint
Sample rate 24 kHz, hop 256 samples (10.67 ms per frame)
Input phoneme ids from a 76-symbol inventory, one stress mark per token
Voices 7 speaker ids (1-6, 8), all from WAXAL's Swahili speakers
Limit 512 tokens per call; 3,000 frames (32 s) of audio
Training steps 55,000 of a planned 104,000

Training

  • Data: swa_tts from google/WaxalNLP (CC-BY-SA-4.0), 24 kHz resampled. 2,497 utterances (~13 h) from 7 speakers passed forced alignment (facebook/wav2vec2-lv-60-espeak-cv-ft, 55% gate); those are the training set. Splits hold out sentence text, not files - WAXAL reuses prompt sentences across recordings.
  • Text front end: a deterministic Swahili letter-to-phone transducer (chβ†’tΚƒ, shβ†’Κƒ, nyβ†’n j, ng'β†’Ε‹, th/dhβ†’ΞΈ/Γ°, penultimate-syllable stress), so no grapheme model or attention is involved. Digits, money, dates and legal citations must be expanded to words before phonemisation; the model itself only ever sees phonemes.
  • Hardware: Kaggle free tier, T4 x2. Stopped at 55k steps when the stage-8 samples were clear to listen to, rather than spending the remaining GPU hours on a step count.

Interface

Inputs (dynamic length on axis 1):

name shape dtype meaning
tokens [1, L] int64 phoneme ids, symbols.json
stress [1, L] int64 0 none, 1 primary, 2 secondary
voice [1] int64 speaker index 0-6
speed [1] float32 duration divisor; 1.2 = 20% faster
pitch_shift [1] float32 log-Hz offset, 0 = unchanged
pitch_range [1] float32 prosody spread multiplier, 1 = unchanged

Outputs:

name shape meaning
audio [1, N] float32 mono at 24 kHz, peak below 0.5
durations [1, L] frames per token; sum(durations) * 256 == N

durations is the alignment: chunk i covers samples [sum(durations[:i]) * 256, sum(durations[:i+1]) * 256). That is exact, not an estimate, which is why word-level highlighting can come from the same forward pass as the audio.

Usage

pip install onnxruntime numpy
python run_onnx.py 0 out.wav

cases.json contains three sentences with pre-computed tokens and stress, so the demo runs without the front end. For arbitrary text, use tiny-voca's Swahili front end (TINY_VOCA_FRONTEND=sw) plus a Swahili verbalizer for digits, dates, money and legal references, then feed the ids and stress arrays.

import numpy as np, onnxruntime as ort
sess = ort.InferenceSession("swahili.onnx", providers=["CPUExecutionProvider"])
audio, durations = sess.run(None, {
    "tokens": np.array([tokens], dtype=np.int64),
    "stress": np.array([stress], dtype=np.int64),
    "voice": np.array([0], dtype=np.int64),
    "speed": np.array([1.0], dtype=np.float32),
    "pitch_shift": np.array([0.0], dtype=np.float32),
    "pitch_range": np.array([1.0], dtype=np.float32),
})

Measured on a Redmi 9A (220233L2G, Android 11, armeabi-v7a)

ONNX Runtime 1.25.0, 4 threads, warm runs:

sentence audio synthesis RTF
Habari za asubuhi! Karibu Kenya. 4.96 s 798 ms 0.161
Leo ni Jumatatu, tarehe kumi na tisa Novemba. 5.77 s 1188 ms 0.206
Bei ya unga ni shilingi elfu moja na mia tano. 5.70 s 1167 ms 0.205

Session creation 5.3 s, PSS 71-75 MB while synthesising. The same export reproduces the PyTorch duration counts and audio length sample for sample; waveforms correlate 0.94-0.98 with the desktop renderer, and two desktop runs of the same input differ by as much.

Known limitations

  • Prototype voice. No native-speaker listening study and no WER/MOS evaluation yet. Expect uneven prosody on long sentences.
  • Each utterance carries a breathy tail (~1.9 s after the last word when synthesised sentence by sentence). Trim to the last word plus 0.25 s; without trimming the reading rate drops to about 70 wpm.
  • Numerals and abbreviations are the caller's job. "2022" must arrive as "elfu mbili na ishirini na mbili"; the model cannot read digits.
  • Read speech of seven speakers. No claim of dialect coverage, code switching, or emotional style. English is not supported.
  • Not a cloning model. There is no speaker encoder; the seven ids are fixed.

Licence and provenance

  • Weights: CC-BY-SA-4.0, inherited from WAXAL's share-alike licence. If you build on these weights, attribute the dataset and share your derivatives alike. The included PyTorch checkpoint is under the same terms.
  • Training data: google/WaxalNLP (swa_tts, CC-BY-SA-4.0). No private or personal recordings were used, and no cloned voice is included; every voice here is a WAXAL speaker.
  • Code: tiny-voca by Nazeem, Apache-2.0 (https://github.com/n4ze3m/tiny-voca).
  • The Swahili front end, verbalizer and on-device app were built for this release and are not part of tiny-voca upstream.

Files

file what
swahili.onnx release graph, internal noise, 21.4 MB
swahili-inference.pt PyTorch checkpoint (step 55,000) for fine-tuning
metadata.json rate, hop, voices, limits, frontend identity
recipe.json training recipe and data hashes
symbols.json the 76 phoneme ids, index = id
config.json upstream config (voices, vocab size)
cases.json three demo sentences with token ids and stress
run_onnx.py renders a case to 24 kHz wav
sample-0/1/2.wav the three cases, rendered by this ONNX
Downloads last month
20
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Dataset used to train adoamesh/tiny-voca-swahili