prathoshap/sushrota-sanskrit-asr · Hugging Face
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#su-%C5%9Brot%C4%81--sanskrit-asr-finetuned-indicconformer-ctc) Su-śrotā — Sanskrit ASR (finetuned IndicConformer-CTC)
A Sanskrit speech-recognition model finetuned for **śāstric and recitational** Sanskrit (chant and prose), where general-purpose Sanskrit ASR degrades badly. It is the model behind **Vāgbodhinī** and **Su-śrotā**, live Sanskrit chant-practice and dictation tools.
- **Base:** AI4Bharat IndicConformer (`EncDecHybridRNNTCTCBPEModel`, ~129 M params). We use the **CTC head** on the Sanskrit token slice.
- **Author:** Prof. Prathosh A P, Indian Institute of Science, Bengaluru.
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#checkpoints-in-this-repo) Checkpoints in this repo
| File | Description | | --- | --- | | `sushrota_sanskrit_asr_v13b.nemo` | **Current / deployed.** Adds consented real-world ("flywheel") data on top of the curated base — far more robust on in-the-wild user recordings. | | `sushrota_sanskrit_asr_v5.nemo` | Prior version, curated data only. Retained for reproducibility. |
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#results) Results
The headline gain of the current model is **real-world robustness**. Evaluated on a leakage-free, 327-clip held-out set of **in-the-wild user recordings** (phones, rooms, varied speakers):
| eval set | metric | v5 (curated only) | **v13b (current)** | | --- | --- | --- | --- | | in-the-wild user audio | CER | 7.70% | **4.36%** (−43%) | | in-the-wild user audio | WER | 45.4% | **30.4%** | | in-the-wild user audio | SN-WER | 20.7–24.4% | **10.8–13.2%** |
On **curated/studio** domains, v13b preserves the base model's quality while adding that robustness:
| domain | CER | WER | SN-WER (sandhi-normalised) | | --- | --- | --- | --- | | Studio held-out (305 clips) | 4.4% | 20.2% | 13.0–15.7% | | Bhāgavata chant (968) | ~6.0% | ~46% | ~22–26% | | Vedānta prose (718) | ~7.2% | ~31% | ~15–19% |
**Note on WER vs SN-WER:** Sanskrit word boundaries are largely orthographic (sandhi fuses words), so raw WER over-penalises boundary disagreements. Roughly _half_ of the WER is spacing; **CER** and **SN-WER** (sandhi-normalised) are the meaningful numbers.
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#training-data) Training Data
The deployed checkpoint (v13b) was trained on **17.4 hours / 6,438 utterances**, combining curated scholar recordings with consented, quality-tiered in-the-wild data.
| Source | Utterances | Hours | Description | | --- | --- | --- | --- | | Scholar recordings (force-aligned) | 2,139 | 6.06 | 21 reciters reading Bhāgavata Purāṇa, Upaniṣad, and stotra texts; forced-aligned to reference | | Recitation corpus | 2,504 | 7.59 | Pre-segmented Upaniṣad (3.0 h), Gītā / Ṛgveda (3.0 h), and additional recitation (1.6 h) | | TTS-speaker augmentation | 637 | 2.00 | Studio/synthetic voices for speaker robustness | | Consented flywheel (in-the-wild) | 1,158 | 1.73 | User recordings from the practice tools, clean tier, ~11 input scripts | | **Total** | **6,438** | **17.37** | |
**Split:** ~90% curated/scholar (15.65 h) + ~10% consented in-the-wild (1.73 h).
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#data-pipeline--quality-tiering) Data pipeline & quality tiering
In-the-wild audio is collected with consent through the practice tools (users read a known reference text) and **automatically quality-graded** at the akṣara level against that reference. Each clip is tiered — `pass` / `override` (clean, trainable), `review` (partial match), or `low` / `unclear` (archived). Only the clean tiers enter training.
The auto-grader was **audited for reliability**: all quarantined `review` clips were re-decoded with the improved model and promoted back to gold only when they scored a perfect akṣara match. Just **4.3%** proved to be model error — confirming the remaining ~96% are genuine reader deviations (mispronunciations, disfluencies, noise), i.e. the grader correctly quarantines real problems rather than mere model disagreements.
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#consented-corpus) Consented corpus
Beyond what is trained on, the flywheel has banked **~28 hours of consented Sanskrit audio from ~1,500 speakers across 11 input scripts** (Devanāgarī, Kannada, Telugu, IAST, and others) — a growing resource for future, more speaker- and script-diverse training rounds.
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#usage) Usage
``` import json, numpy as np, torch, soundfile as sf import nemo.collections.asr as na
M = na.models.EncDecHybridRNNTCTCBPEModel.restore_from("sushrota_sanskrit_asr_v13b.nemo").eval()
OFF, V, BLANK = 4096, 256, 5632 # Sanskrit token slice of the aggregate vocab def greedy(wav): # wav: 16 kHz mono float32 sig = torch.tensor(wav).unsqueeze(0); sl = torch.tensor([len(wav)]) with torch.no_grad(): enc, _ = M.forward(input_signal=sig, input_signal_length=sl) lp = M.ctc_decoder(encoder_output=enc)[0].cpu().numpy() cols = [BLANK] + list(range(OFF, OFF + V)) P = lp[:, cols]; P = P - (P.max(1, keepdims=True) + np.log(np.exp(P - P.max(1, keepdims=True)).sum(1, keepdims=True))) # re-log_softmax on slice ids = P.argmax(1) sub = M.tokenizer.tokenizers_dict["sa"] # Sanskrit SentencePiece out, prev = [], -1 for i in ids: i = int(i) if i != prev and i != 0: out.append(sub.ids_to_tokens([i - 1])[0]) prev = i return "".join(out).replace("▁", " ").strip()
wav, sr = sf.read("clip.wav", dtype="float32") print(greedy(wav)) ```
The model is an aggregate multilingual IndicConformer; decode on the Sanskrit slice as above.
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#provenance--consent) Provenance & consent
Scholar recordings were contributed for the purpose of building this model. In-the-wild clips are collected only with explicit user consent and store no raw IP or personal identifiers (an anonymous per-session id only). Individual reciter names are not published.
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#related) Related
- **Vāgbodhinī** (chant-practice tool + full experiment report): https://github.com/prathoshap/sushrota-sanskrit-asr
- **Vāgdhenu** (metre-aware Sanskrit chant TTS): `prathoshap/vagdhenu`
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#license) License
Finetuned from AI4Bharat's IndicConformer — please observe the base model's license terms.
[](https://huggingface.co/prathoshap/sushrota-sanskrit-asr#citation) Citation
> Prathosh A P, _Su-śrotā: Scholar-grade Sanskrit ASR and metre-aware chant practice_, Indian Institute of Science, Bengaluru, 2026.