Bulk Runner & Evaluator

Compare Telugu Speech Recognition

In this bulk run example, we compare 3 different Gooey.AI speech recognition + translations workflows (https://gooey.ai/speech), each of which uses a different AI speech model - OpenAI's Whisper Large vs Bhashini's Fine-tuned Telugu model vs Google's USM Chirp. You can see from the results, that model #3 - Google's USM - actually performs best, based on the quality of the Telugu transcription and English translation.

Documents

🔗telugu_audio.csv

Ulangizi Copilot Eval

Testing AI bots and copilots (https://gooey.ai/copilot) in practice is complex because the answer a bot will give is affected not just by the user's last question but also by the conversation history that preceded that question. Hence, this bulk example shows you how to format your CSV to include a bot/user conversation history before each question is asked. The output columns also show which knowledge documents were returned as results that the bot used in trying to formulate its answer.

Documents

🔗https://docs.google.com/spreadsheets/d/1kOng5UJJ8yvWfgKbWEjk3mSNNbBUrPb6SiE2srlOLBM/edit?usp=sharing

Copilot Message History Eval

Testing AI bots and copilots (https://gooey.ai/copilot) in practice is complex because the answer a bot will give is affected not just by the user's last question but also by the conversation history that preceded that question. Hence, this bulk example shows you how to format your CSV to include a bot/user conversation history before each question is asked. The output columns also show which knowledge documents were returned as results that the bot used in trying to formulate its answer.

Documents

🔗bulk-runner-0-3.csv

Bulk Chichewa ASR and Translation

Here we input a set of 3 6-9 minute long mp3s of Chichewa videos, run them through google's USM for transcription and translation. The results are OK....

Documents

🔗https://docs.google.com/spreadsheets/d/1PkyAW_gJgnGUR4hmhP6Hl3kFThPIPt4XzgST1cxyCMA/edit#gid=1695511050

Compare Chichewa ASR and Translation

Bulk Runner for different dense embeddings weightages

This example takes a CSV with URLs and image descriptions and then runs our artistic QR Code maker over each of them, generating two QR codes for each URL + description pair. This is great if you want to create 100s of art QR Codes in one huge batch.

Documents

🔗DENSE-weightage-copilot - Sheet1.csv

Compare Anime vs Pixar Profile Pix style

This bulk run compares two different AI Image editor workflows - one that takes profile pictures and renders them in an anime vs Pixar style. Swap out the CSV with your own list of profile photo URLs to test even more. Swap out the workflow URLs with different Image prompts to test out new styles.

About the Bulk Runner: Which AI model actually works best for your needs? Upload your own data and evaluate any Gooey.AI workflow, LLM or AI model against any other. Great for large data sets, AI model evaluation, task automation, parallel processing and automated testing. To get started, paste in a Gooey.AI workflow, upload a CSV of your test data (with header names!), check the mapping of headers to workflow inputs and tap Submit. More tips in the Details below.

Documents

🔗https://docs.google.com/spreadsheets/d/1vrR-QR-7i9vb-9xatdlC4k3rAR47yP5VamwA1db7jgA/edit?usp=sharing

Bulk Create Art QR codes

This example takes a CSV with URLs and image descriptions and then runs our artistic QR Code maker over each of them, generating two QR codes for each URL + description pair. This is great if you want to create 100s of art QR Codes in one huge batch.

Documents

🔗qrcodes.csv