(26 Qs Updated December 2025)

Benchmark Results

Sunbird v2 + Gemini 3 Pro leads!

The top‑achieving Kikuyu benchmark in this workflow is Workflow 3 – “Kikuyu (Akera + Gem3pro)”, with an average similarity score of 0.78 against the reference transcripts. This setup combines Akera’s Kikuyu‑focused ASR with Gemini 3 Pro for post‑processing/translation, and it consistently delivers the closest match to the ground‑truth text across the 26 audio clips—slightly outperforming the Omnilingual + Gemini and Omnilingual + MT + GPT‑5.1 pipelines, and dramatically better than the baseline GPT‑Realtime audio route in this test set.

Info

This workflow benchmarks multiple Kikuyu speech-to-text pipelines on the same audio dataset. It compares GPT‑4o audio, Akera GPT‑5.1, Gemini 3 Pro, Omni and Kikuyu → English pipelines with Google Machine Translation.

Workflows covered

• GPT‑4o Audio – Kikuyu ASR
• Akera + MT + Gemini3 Pro
• Akera + GPT‑5.1 – Kikuyu-focused ASR & NLP
• Omnilingual + Gemini3 Pro– Kikuyu → English
• Omnilingual + MT + Gemini3 Pro – multimodal Kikuyu ASR
• Omnilingual + MT + GPT-5.1

Data & Evaluation

Kikuyu audio & reference transcripts from Google Sheets
Automated evaluator: compare-output-text-from-input_audio for text similarity and quality scoring.

Ideal for teams comparing Kikuyu speech recognition, Kikuyu audio transcription, and Kikuyu → English translation for call centers, media, education, and African-language AI applications.

Keywords: Kikuyu speech-to-text, Kikuyu ASR, Kikuyu audio transcription, Kikuyu to English translation, Akera AI Swahili, GPT‑4o audio Kikuyu, Gemini Pro Kikuyu transcription, Google MT Kikuyu.

(3 Qs Updated December 2025)

Swahili (Omnilingual + GPT-5 + Google MT) (A2T)

A simple bot to answer Qs from small shareholder farmers.

A test number for MBaza ASR + GPT5-mini for Kinyarwanda via Gooey's LiveKit-based voice system. +19288000048

Kinyarwanda using GPT-realtime on +14255173811. Using prompt following OpenAI cookbook guidelines

Here we compare the top speech recognition, LLM and machine translation models for Kinyarwanda. In short, unfortunately the realtime OpenAI models (GPT4o-Audio and GPT-realtime) perform badly while leveraging dedicated ASR models (MMS, Sunbird, MBaza) in conjunction with GPT5 and Google 2.5 Pro appears to work reasonably well.

A version that attempts to use the same models as ChatGPT for Kinyarwanda as of Sept 15, 2025. GPT-4o Audio (the latest published audio models) + GPT5.

Here we compare the top speech recognition, LLM and machine translation models for Kinyarwanda. In short, unfortunately the realtime OpenAI models (GPT4o-Audio and GPT-realtime) perform badly while leveraging dedicated ASR models (MMS, Sunbird, MBaza) in conjunction with GPT5 and Google 2.5 Pro appears to work reasonably well.