Gates Foundation

gatesfoundation

A workspace for the Gates Foundation DPI, FairFoward and Gooey teams focused on evals for low-resource languages plus the home of our Agriculture advisory work e.g. https://gooey.ai/ageval

489 Public Workflows
10 Members

(Updated Dec 2025)
This workflow benchmarks multiple English speech‑to‑text pipelines on the same audio dataset. It compares GPT‑4o audio, realtime GPT, GPT‑5.2, Gemini 3 Pro, Llama 4, and DeepSeek‑32 based transcription workflows.

Workflows covered
Each row of your dataset (Google Sheet) with an input_audio URL is sent to all of these Gooey workflows:

GPT‑4o Audio – English ASR

URL: https://gooey.ai/copilot/0-gpt-4oaudio-english-a2t-cumo9m8mbssd/
Direct audio→text via GPT‑4o’s native audio understanding.
GPT Realtime – Streaming English Transcription

URL: https://gooey.ai/copilot/1-gpt-realtime-english-a2t-xxisb52q5i8l/
Simulates realtime / streaming ASR for latency‑sensitive use‑cases.
GPT‑5.2 – English Audio→Text Pipeline

URL: https://gooey.ai/copilot/2-gpt-52-english-a2t-j48rl7w31egx/
Uses GPT‑5.2 as the primary model for transcription and light cleanup.
Gemini 3 Pro – English Audio→Text

URL: https://gooey.ai/copilot/3-gemini3pro-english-a2t-22ehqxbjujdn/
Google Gemini 3 Pro based transcription workflow for English audio.
Llama 4 – English ASR + LLM Post‑Processing

URL: https://gooey.ai/copilot/4-llama4-english-a2t-i2wytf132u11/
Llama 4 used for transcription and/or normalization of English speech.
DeepSeek‑32 – English Audio Transcription

URL: https://gooey.ai/copilot/5-deepseek32-english-a2t-iy8blj05sfgr/
DeepSeek‑32 model pipeline for English audio→text.

(Updated Dec 2025)

This workflow benchmarks multiple Swahili (Kiswahili) speech-to-text pipelines on the same audio dataset. It compares GPT‑4o audio, Jacaranda GPT‑5.1, Gemini 3 Pro, Omni and Swahili → English pipelines with Google Machine Translation.

Workflows covered

  • GPT‑4o Audio
  • GPT Realtime
  • Jacaranda + GPT‑5.1
  • Jacaranda + Gemini 3 Pro
  • Jacaranda GPT‑5.1 + Google MT
  • Omni GPT‑5.1 + Google MT
  • Omni + Gemini 3 Pro
  • Omni + Gemini + Google MT

Data & Evaluation

Swahili audio & reference transcripts from Google Sheets
Automated evaluator: compare-output-text-from-input_audio for text similarity and quality scoring.

Ideal for teams comparing Swahili speech recognition, Swahili audio transcription, and Swahili → English translation for call centers, media, education, and African-language AI applications.

Keywords: Swahili speech-to-text, Kiswahili ASR, Swahili audio transcription, Swahili to English translation, Jacaranda AI Swahili, GPT‑4o audio Swahili, Gemini Pro Swahili transcription, Google MT Swahili.

Here we compare the the top Kikuyu ASR (Sunbird v2), LLM (OpenAI realtime models, GPT5 Gemini 2.5 Pro) and MT (Google Translate and the LLMs to directly understand Kikuyu text). Early results show Sunbird v2 + Gemini 2.5 Pro.

Here we compare the the top Kikuyu ASR (Akera, MMS), LLM (OpenAI realtime models, GPT5, GPT5-mini and Gemini 2.5 Pro) and MT (Google Translate, GhanaNLP and the LLMs to directly understand Kikuyu text). Early results show Sunbird v2 + Gemini 2.5 Pro works best.