Gates Foundation

gatesfoundation

A workspace for the Gates Foundation DPI, FairFoward and Gooey teams focused on evals for low-resource languages plus the home of our Agriculture advisory work e.g. https://gooey.ai/ageval

332 Public Workflows
8 Members

A simple bot to answer Qs from small shareholder farmers.

6mo ago

372 runs

Public

A simple bot to answer Qs from small shareholder farmers.

6mo ago

378 runs

Public

A test number for MBaza ASR + GPT5-mini for Kinyarwanda via Gooey's LiveKit-based voice system. +19288000048

test gemini

9mo ago

76 runs

Public

Kinyarwanda using GPT-realtime on +14255173811. Using prompt following OpenAI cookbook guidelines

Updated to use long prompt.

9mo ago

83 runs

Public

Here we compare the top speech recognition, LLM and machine translation models for Kinyarwanda. In short, unfortunately the realtime OpenAI models (GPT4o-Audio and GPT-realtime) perform badly while leveraging dedicated ASR models (MMS, Sunbird, MBaza) in conjunction with GPT5 and Google 2.5 Pro appears to work reasonably well.

A version that attempts to use the same models as ChatGPT for Kinyarwanda as of Sept 15, 2025. GPT-4o Audio (the latest published audio models) + GPT5.

Added TTS

9mo ago

Public

Here we compare the top speech recognition, LLM and machine translation models for Kinyarwanda. In short, unfortunately the realtime OpenAI models (GPT4o-Audio and GPT-realtime) perform badly while leveraging dedicated ASR models (MMS, Sunbird, MBaza) in conjunction with GPT5 and Google 2.5 Pro appears to work reasonably well.

Which of the latest models best understand Swahili questions (as WhatsApp audio notes) and can provide an English? GPT-realtime, GPT4o realtime, Jacaranda(ASR) + GPT5 (LLM as MT) and Jacarandra + GPT5 + Google Translate are compared.

Removed extra rows and columns

9mo ago

Public

Here we compare the top speech recognition, LLM and machine translation models for Kinyarwanda. In short, unfortunately the realtime OpenAI models (GPT4o-Audio and GPT-realtime) perform badly while leveraging dedicated ASR models (MMS, Sunbird, MBaza) in conjunction with GPT5 and Google 2.5 Pro appears to work reasonably well.

9mo ago

Public

This workflow is designed specifically to measure end-to-end latency for voice-based AI interactions, focusing on benchmarking system response times rather than providing full conversational answers. Incoming audio samples (in Kikuyu or other selected languages) are transcribed, processed by an AI assistant with a maximum output of 10 tokens, and then synthesized back to audio. The workflow compares two different Kikuyu audio-to-audio (A2A) translation pipelines by processing input samples from a Google Sheet and logging runtime, price, and output URLs for each. This setup enables reliable benchmarking and optimization of transcription, AI processing, and text-to-speech components, helping teams evaluate latency and cost across different ASR models for the Kikuyu language.

🦾

9mo ago

Public

Here we compare the top speech recognition, LLM and machine translation models for Kinyarwanda. In short, unfortunately the realtime OpenAI models (GPT4o-Audio and GPT-realtime) perform badly while leveraging dedicated ASR models (MMS, Sunbird, MBaza) in conjunction with GPT5 and Google 2.5 Pro appears to work reasonably well.