Ulangizi AI with Vision - v12 - Gemini Pro 1.0 Vision

A Bot for Opportunity.org that gives advice to Malawi farmers and extension agents based on a guideline PDF from the Ministry of Agriculture. Support incoming pictures of sick plants, understands English and Chickiwa and is connected to WhatsApp bots as well.

📝 Instructions

🧠 Language Model

Gemini 1.0 Pro Vision (Google)

📄 Knowledge

Submit Links in Bulk

💪 Capabilities

🦻 Speech Recognition & Translation

🗣️ Text to Speech & Lipsync

Text-to-Speech Provider

Azure Text-to-Speech

Azure TTS Voice name

Imani - English (Tanzania)

🫦 Add Lipsync Video

🩻 Photo & Document Intelligence

🧩 Developer Tools and Functions

⌥ Variables

final_search_query

string

Template variable

⚙️ Settings

Microsoft Server Speech Text to Speech Voice (en-TZ, ImaniNeural):

Name: Imani
Gender: Female
Locale: English (Tanzania)
Locale Code: en-TZ
Sample Rate: 48000 Hz
Voice Type: Neural
Words Per Minute: 142

See all the supported languages and voices here.

📄 Knowledge Base

👩‍🏫 Search Instructions

How should the LLM interpret the results from your knowledge base?

Citation Style

Plain Text / WhatsApp Symbols + Footnotes

🔗 Shorten Citation URLs

Create Synthetic Data

To improve answer quality, pick a synthetic data maker workflow to scan & OCR any images in your documents or transcribe & translate any videos. It also can synthesize a helpful FAQ. Adds ~2 minutes of one-time processing per file.

———

Advanced Settings

In general, you should not need to adjust these.

👁‍🗨 Conversation Summarization

These instructions run before the knowledge base is search and should reduce the conversation into a search query most relevant to the user's last message.

🔑 Keyword Extraction

Instructions to create a query for keyword/hybrid BM25 search. Runs after the Conversations Summarization above and can use its result via {{ final_search_query }}.

✏ Embeddings Model

Text Embedding 3 Large (OpenAI)

Dense Embeddings Weightage

Weightage for dense vs sparse embeddings. 0 for sparse, 1 for dense, 0.5 for equal weight.
Generally speaking, dense embeddings excel at understanding the context of the query, whereas sparse vectors excel at keyword matches.

Max Citations

The maximum number of document search citations.

Max Snippet Words

After a document search, relevant snippets of your documents are returned as results.
This setting adjusts the maximum number of words in each snippet (tokens = words * 2).
A high snippet size allows the LLM to access more information from your document results, at the cost of being verbose and potentially exhausting input tokens (which can cause a failure of the copilot to respond).

Snippet Overlap Ratio

Your knowledge base documents are split into overlapping snippets.
This settings adjusts how much those snippets overlap (overlap tokens = snippet tokens / overlap ratio).
In general you shouldn't need to adjust this.

🔠 Language Model Settings

Avoid Repetition

Max Output Tokens

The maximum number of tokens to generate in the completion. Increase to generate longer responses.

Creativity (aka Sampling Temperature)

Higher values allow the LLM to take more risks. Try values larger than 1 for more creative applications or 0 to ensure that LLM gives the same answer when given the same user input.

Answer Outputs

How many answers should the copilot generate? Additional answer outputs increase the cost of each run.

Run cost = 5 credits

Breakdown: 1 (Gemini 1.0 Pro Vision (Google)) + 3/run

By submitting, you agree to Gooey.AI's terms & privacy policy.

Show Raw Output

Generated in 0.0s on

...

ℹ️ Details

How to Use This Recipe

🙋🏽‍♀️ Need more help? Join our Discord