Gates Foundation

gatesfoundation

A workspace for the Gates Foundation DPI, FairFoward and Gooey teams focused on evals for low-resource languages plus the home of our Agriculture advisory work e.g. https://gooey.ai/ageval

489 Public Workflows
10 Members

💬

6mo ago

104 runs

Public

(25 Qs Updated 16 April 2026)
This page shows a test of many Kinyarwanda speech‑to‑text systems.
kinyarwanda
For each system, we play the same Kinyarwanda audio and capture its text output.
We then compare that text to a trusted reference answer and give a score between 0 and 1.

#WorkflowAccuracy (Mean)Latency (Median, s)
2Mbaza+Gem3.1pro (A2T)0.8910.18
6Omni+Gem3.1pro (A2T)0.9011.13
9Mbaza+Gem3.1flash-lite (A2T)0.873.88
10Omni+Deepseek3.20.868.63
11Omni+KimiK2.50.8212.61
12Omni+MiniMax2.70.626.82
13Omni+GLM5.10.7910.81
14Mbaza+MiniMax2.70.505.13
15Mbaza+KimiK2.50.8213.83
16Mbaza+GPTOSS 120B0.836.50
17Mbaza+Qwen3.6+0.849.54

A higher score means the system output is closer to the reference text and usually more accurate.

You can use these scores to:

  • See which system gets the best score
  • Compare different models and pipelines side by side
  • Choose the best system for your app, research, or product
  • Download all results for deeper analysis

🦾

6mo ago

Public

💬

Share settings updated

7mo ago

182 runs

Public

A bulk evaluator workflow that compares AI-generated answers (copilot responses) to a set of golden reference answers. Requires input data columns: "input_prompt" (the question/task) and "reference_answer" (the ideal response). The workflow uses custom evaluation prompts to compare outputs, scoring them for accuracy and penalizing hallucinations. Aggregates results to provide an overall performance metric for your AI answers.

⚖️

Share settings updated

7mo ago

Public

(20 Qs Updated Jan 2025)

This page shows a test of many Hindi speech‑to‑text systems and, in some cases, Hindi → English translation pipelines.
hindi benchmark

We use the same Hindi audio clips for every system. Then we compare each system’s text output to a reference answer and give it a score between 0 and 1.
A higher score means the system is closer to the reference text and usually more accurate.

WorkflowAccuracy (mean score)Latency (median)
0 GPT4oAudio0.777.33
1 GPTRealtime0.736.64
2 GPT5.10.925.69
3 GPT4.10.905.67
4 Gemini 3 Pro0.969.88
5 Gemini 3 Flash0.927.58
6 Sarvam.AI0.555.98
7 Omnilingual+GPT5-mini0.969.00
8 Omnilingual+Gemini 3 Pro0.9310.14
9 Omnilingual+Gemini 3 Flash0.917.60
10 MMS+GoogMT+GPT4.10.914.94

On this page you can:

  • See which Hindi system or pipeline gets the best score
  • Compare different Hindi ASR and Hindi→English models side by side
  • Choose the best system for your app, call center, research, or product
  • Download all results for deeper analysis and custom reporting

🦾

Share settings updated

8mo ago

Public

💬

Share settings updated

9mo ago

101 runs

Public

💬

Share settings updated

9mo ago

51 runs

Public

(Updated Jan 2025)
This page shows a test of many English speech‑to‑text systems.
English Benchmark
We use the same English audio for every system. Then we compare each system’s text to a reference answer and give it a score between 0 and 1.
A higher score means the system is closer to the reference text and usually more accurate.

Ranking

WorkflowAccuracy (mean score)Latency (median, s)
GPT-4o Audio0.937.75
GPT-Realtime0.937.70
GPT‑5.20.915.99
Gemini 3 Pro0.8610.16
Llama 40.915.77
DeepSeek 3.20.886.17
Gemini 3 Flash0.897.77
GPT‑4.10.926.02

With this benchmark, you can:

  • See which system gets the best score
  • Compare different models and pipelines side by side
  • Choose the best system for your app, research, or product
  • Download all results for deeper analysis

9mo ago

Public