Gates Foundation

gatesfoundation

A workspace for the Gates Foundation DPI, FairFoward and Gooey teams focused on evals for low-resource languages plus the home of our Agriculture advisory work e.g. https://gooey.ai/ageval

332 Public Workflows
8 Members

4 Jacaranda + GPT5+Google MT) (A2T) for Swahili speakers

5mo ago

445 runs

Public

Compares Gemini3, GPT5.2, KissanAI, LLAMA4 Maverick, Sarvam.AI, GPT4o and AgriLLM (Qwen3) for their responses being similar to our golden QnA of common small shareholder farmer questions and answers (in English) from ClearGlobal and Opportunity Intl. What makes a good golden eval QnA?
Results

5mo ago

Public

A simple bot to answer Qs from small shareholder farmers.

5mo ago

385 runs

Public

A simple bot to answer Qs from small shareholder farmers.

5mo ago

Public

A simple bot to answer Qs from small shareholder farmers.

updated model KimiK2

5mo ago

Public

A simple bot to answer Qs from small shareholder farmers.

Changed to Dhenu3 model

5mo ago

255 runs

Public

(Updated Dec 2025)
This workflow benchmarks multiple English speech‑to‑text pipelines on the same audio dataset. It compares GPT‑4o audio, realtime GPT, GPT‑5.2, Gemini 3 Pro, Llama 4, and DeepSeek‑32 based transcription workflows.

Workflows covered
Each row of your dataset (Google Sheet) with an input_audio URL is sent to all of these Gooey workflows:

GPT‑4o Audio – English ASR

URL: https://gooey.ai/copilot/0-gpt-4oaudio-english-a2t-cumo9m8mbssd/
Direct audio→text via GPT‑4o’s native audio understanding.
GPT Realtime – Streaming English Transcription

URL: https://gooey.ai/copilot/1-gpt-realtime-english-a2t-xxisb52q5i8l/
Simulates realtime / streaming ASR for latency‑sensitive use‑cases.
GPT‑5.2 – English Audio→Text Pipeline

URL: https://gooey.ai/copilot/2-gpt-52-english-a2t-j48rl7w31egx/
Uses GPT‑5.2 as the primary model for transcription and light cleanup.
Gemini 3 Pro – English Audio→Text

URL: https://gooey.ai/copilot/3-gemini3pro-english-a2t-22ehqxbjujdn/
Google Gemini 3 Pro based transcription workflow for English audio.
Llama 4 – English ASR + LLM Post‑Processing

URL: https://gooey.ai/copilot/4-llama4-english-a2t-i2wytf132u11/
Llama 4 used for transcription and/or normalization of English speech.
DeepSeek‑32 – English Audio Transcription

URL: https://gooey.ai/copilot/5-deepseek32-english-a2t-iy8blj05sfgr/
DeepSeek‑32 model pipeline for English audio→text.

(Updated Dec 2025)

This workflow benchmarks multiple Swahili (Kiswahili) speech-to-text pipelines on the same audio dataset. It compares GPT‑4o audio, Jacaranda GPT‑5.1, Gemini 3 Pro, Omni and Swahili → English pipelines with Google Machine Translation.

Workflows covered

  • GPT‑4o Audio
  • GPT Realtime
  • Jacaranda + GPT‑5.1
  • Jacaranda + Gemini 3 Pro
  • Jacaranda GPT‑5.1 + Google MT
  • Omni GPT‑5.1 + Google MT
  • Omni + Gemini 3 Pro
  • Omni + Gemini + Google MT

Data & Evaluation

Swahili audio & reference transcripts from Google Sheets
Automated evaluator: compare-output-text-from-input_audio for text similarity and quality scoring.

Ideal for teams comparing Swahili speech recognition, Swahili audio transcription, and Swahili → English translation for call centers, media, education, and African-language AI applications.

Keywords: Swahili speech-to-text, Kiswahili ASR, Swahili audio transcription, Swahili to English translation, Jacaranda AI Swahili, GPT‑4o audio Swahili, Gemini Pro Swahili transcription, Google MT Swahili.