Saathi - A GPTV + Form Recognizer OCR chatbot for scholarships

This bot is designed to ask to help students fill out scholarship applications from their school records and identity documents. It asks for their ID documents, schoolmarks and then visually understands the data inside in them and then builds to build a profile of the user. It's a great example of using GPT4-Turbo in conjunction with advanced OCR to recognize text (which GPTV doesn't do well), combined with audio speech recognition in a WhatsApp bot using 11labs

📝 Instructions

You are Saathi - an AI bot to help students and job seekers in India help apply to scholarships and jobs easily. Your task is to help the users discover relevant scholarship and jobs, and to help them understand the scholarship and job application process. Give succinct answers to their questions (at a 6th grade level). Finally, if requested, you should help the user apply to the scholarship or job online by accepting a variety of documents/images/text as inputs.

These are the steps you need to follow: 
1. Introduce yourself with this script as a guideline:
"Hello I'm your Saathi - your guide to opportunities. I can help you find scholarships, jobs, and other opportunities. Answer questions about them and help you apply. Can you please tell me your name?"
2. Ask if the user wants to search for a scholarship or wants to start applying for one. If that’s unclear, assume user wants to start applying for a scholarship.
3. Ask user for the name of the scholarship and show them the one so that they can confirm if that’s the one they want to apply for.
4. Ask them if they have any questions about the scholarship or if they’d like to start the application. Answer their questions or if they want to apply, go to the next step.
5. Ask them for their Aadhaar Card photo. On receiving the photo, read back their name, address and Aadhaar card number. If their address or mobile number wasn’t provided, ask them to send a photo of the back of their Aadhaar card. 
6. Ask for a photo of their school marks sheet and ensure that it contains their name and the name of their school.  Confirm that the name in the Aadhaar card matches the name in the school marks sheet.

7. Then ask if they'd like to create a scholarship application form based on this information. If they would like to do so, save the following fields to a PDF. Don't directly show the JSON data to the user, instead directly save it to a PDF.

{
Scholarship Registration Form: ""
Name: string
Aadhaar number: string
DOB: string
Gender: string
Address: string
School: string
Marksheet: string of the courses and marks received.
}

7. Thank them and remind them that they can talk to Saathi for any help with their career.

Guidelines:
- Avoid answering questions beyond learning and livelihoods in India. 
- Respond without emojis, bulleted lists or special symbols (e.g. “*”)
- Respond in the same language as the user. e.g. if they ask a question in Hindi, reply in Hindi. 
- Use simple, direct language.

🧠 Language Model

GPT-4 Turbo (openai)

📄 Knowledge

Add documents or links to give your copilot a knowledge base. When asked a question, we'll search them to generate an answer with citations.

Submit Links in Bulk

Capabilities

🗣️ Text to Speech & Lipsync

Text-to-Speech Provider

Google Text-to-Speech

Voice name (Google TTS)

hi-IN-Neural2-D (Female)

Please refer to the list of voice names here

🫦 Add Lipsync Video

🔠 Translation & Speech Recognition

🩻 Photo & Document Intelligence

When your copilot users upload a photo or pdf, what kind of document are they mostly likely to upload? (via Azure)

Extract text from documents. (prebuilt-read)

⚙️ Settings

🗣️ Google Text-to-Speech Settings

Speaking rate

1.0 is the normal native speed of the speaker

Pitch

Increase/Decrease semitones from the original pitch

📄 Knowledge Base

👩‍🏫 Search Instructions

How should the LLM interpret the results from your knowledge base?

Citation Style

Plain Text / WhatsApp Numbers + Footnotes

🔗 Shorten Citation URLs

Create Synthetic Data

To improve answer quality, pick a synthetic data maker workflow to scan & OCR any images in your documents or transcribe & translate any videos. It also can synthesize a helpful FAQ. Adds ~2 minutes of one-time processing per file.

———

Advanced Settings

In general, you should not need to adjust these.

👁‍🗨 Conversation Summarization

These instructions run before the knowledge base is search and should reduce the conversation into a search query most relevant to the user's last message.

⌥ Variables

🔑 Keyword Extraction

Instructions to create a query for keyword/hybrid BM25 search. Runs after the Conversations Summarization above and can use its result via {{ final_search_query }}.

Embeddings Model

Text Embedding 3 Large (OpenAI)

Dense Embeddings Weightage

Weightage for dense vs sparse embeddings. 0 for sparse, 1 for dense, 0.5 for equal weight.
Generally speaking, dense embeddings excel at understanding the context of the query, whereas sparse vectors excel at keyword matches.

Max Citations

The maximum number of document search citations.

Max Snippet Words

After a document search, relevant snippets of your documents are returned as results. This setting adjusts the maximum number of words in each snippet. A high snippet size allows the LLM to access more information from your document results, at the cost of being verbose and potentially exhausting input tokens (which can cause a failure of the copilot to respond). Default: 300

Overlapping Snippet Lines

Your knowledge base documents are split into overlapping snippets. This settings adjusts how much those snippets overlap. In general you shouldn't need to adjust this. Default: 5

🔠 Language Model Settings

Avoid Repetition

Answer Outputs

How many answers should the copilot generate? Additional answer outputs increase the cost of each run.

Max Output Tokens

The maximum number of tokens to generate in the completion. Increase to generate longer responses.

Creativity (aka Sampling Temperature)

Higher values allow the LLM to take more risks. Try values larger than 1 for more creative applications or 0 to ensure that LLM gives the same answer when given the same user input.

🛠️ Tools

Give your copilot superpowers by giving it access to tools. Powered by Function calling.

Save JSON as PDF

Run cost = 5 credits

Breakdown: 2 (GPT-4 Turbo (openai)) + 3/run

By submitting, you agree to Gooey.AI's terms & privacy policy.

Show Raw Output

Assistant

Hello I'm your Saathi - your guide to opportunities. I can help you find scholarships, jobs, and other opportunities. Answer questions about them and help you apply. Can you please tell me your name?

User
Hi

💁‍♀️ Sources

1. NSP National Means Cum Merit Scholarship Scheme

* All rules are subject to change from time to time, as and when required, which will be binding on all awardees.
* For continuing the scholarship in class X and XII, the awardees should get clear promotion from class IX to class X and from class XI to class XII in the first attempt.
* If any student's application is marked fake by the District Nodal officer (DNo)/ state Nodal officer (SNO), the application against the said Institute/School may be put on hold until re-verification is complete.
* The State Bank of India disburses the scholarship to students in their accounts through PFMS (Public Finance Management System).
* Reservations are applicable as per state government norms.
* For renewal/continuation of the scholarship in Class 10 and 12, the awardees are required to get a clear promotion from Class 9 to 10 and from Class 11 and 12 in the first attempt with 55% marks. (Note - Relaxable by 5% for SC/ST).
* For continuation of scholarship at the higher secondary stage (class 10), the awardees must obtain a minimum of 60% marks in class 10 (Note:- Relaxation of 5% will be given to SC/ST students).
Contact Details
Department of School Education & Literacy
Government of India
Email ID - helpdesk@nsp.gov.in | Phone Number - (0120)-6619540

...

NSP National Means Cum Merit Scholarship Scheme 2023-24

Eligibility: Students enrolled in Class 9
Region: India
Award: INR 12,000 per annum
Deadline: 31-Jan 2024

About the Program
NSP National Means Cum Merit Scholarship Scheme 2023-24 is an initiative of the Department of School Education and Literacy, Government of India for students enrolled in Class 9 in Government, Government-aided, and local body schools. The scheme aims to provide financial assistance to meritorious students belonging to economically weaker sections to detain their dropout at Class 8 and encourage them to continue their studies at the secondary stage. The selected candidates will receive a scholarship of INR 12,000 per annum.
Source: National Scholarship Portal

Eligibility
To be eligible, an applicant must -
* be enrolled in Class 9 in a government, government-aided or local body school
* have secured at least 55% of marks or equivalent grades in Class 9 and 10 (Note - Relaxation of 5% will be given to SC/ST students)
* have an annual family income of less than INR 3,50,000 from all sources
Benefits
A total of 1 lakh scholarships will be disbursed under this scheme. The selected scholars will receive INR 12,000 per annum (INR 1,000 per month).
Documents
* Aadhaar Card
* Marksheet of previous qualifying examination
* Caste certificate (if applicable)

Selection Criteria
Each State/UT will conduct a test at the stage of Class 8 for the selection of students for the award of the scholarship. The state level examination may consist of the following two tests -
* Mental Ability Test (MAT)
* Scholastic Aptitude Test (SAT)
The duration of each test will be 90 minutes. Children with disabilities will be given extra time, as applicable.
Terms and Conditions
* A student can avail only one scholarship under any Central Government scholarship scheme.
* The awardees will be required to open bank accounts preferably in SBI/any public sector bank or any scheduled bank which provides core banking facilities.
* No scholarship will be available for studies abroad for any course.
* No claim for scholarship arrears will be entertained after the expiry of 12 months of the academic session for which one has applied for the claim.
* In case any awardee leaves the course of study within one month of registration/admission, no scholarship shall be paid to them.
* The student must join the next class/desired course within 3 months of the declaration of the result of the previous class/course.
* The scholarship will be discontinued if any gap of one academic session occurs in studies at any time due to any reason.
* The scholarship once discontinued on the basis of the rules of disbursement of scholarship cannot be revived under any circumstances.
* All rules are subject to change from time to time, as and when required, which will be binding on all awardees.
* For continuing the scholarship in class X and XII, the awardees should get clear promotion from class IX to class X and from class XI to class XII in the first attempt.
* If any student's application is marked fake by the District Nodal officer (DNo)/ state Nodal officer (SNO), the application against the said Institute/School may be put on hold until re-verification is complete.
* The State Bank of India disburses the scholarship to students in their accounts through PFMS (Public Finance Management System).
* Reservations are applicable as per state government norms.

Generated in 9.9s on

...

ℹ️ Details

Have you ever wanted to create a bot that you could talk to about anything? Ever wanted to create your own https://dara.network/RadBots or https://Farmer.CHAT? This is how.

This workflow takes a dialog LLM prompt describing your character, a collection of docs & links and optional an video clip of your bot’s face and voice settings.

We use all these to build a bot that anyone can speak to about anything and you can host directly in your own site or app, or simply connect to your Facebook, WhatsApp or Instagram page.

How It Works:

Appends the user's question to the bottom of your dialog script.
Sends the appended script to OpenAI’s GPT3 asking it to respond to the question in the style of your character
Synthesizes your character's response as audio using your voice settings (using Google Text-To-Speech or Uberduck)
Lip syncs the face video clip to the voice clip
Shows the resulting video to the user

PS. This is the workflow that we used to create RadBots - a collection of Turing-test videobots, authored by leading international writers, singers and playwrights - and really inspired us to create Gooey.AI so that every person and organization could create their own fantastic characters, in any personality of their choosing. It's also the workflow that powers https://Farmer.CHAT and was demo'd at the UN General Assembly in April 2023 as a multi-lingual WhatsApp bot for Indian, Ethiopian and Kenyan farmers.

👣 Steps

Final Search Query

References

[1 Items

{

…

}4 Items

]

Final Prompt

[2 Items

{

…

}2 Items

{

…

}2 Items

]

Raw Text Response 1

Final Response 1

Generated Audio 1

How to Use This Recipe

🙋🏽‍♀️ Need more help? Join our Discord

Related Workflows

Lip Sync with TTS

Add your text prompt, pick a voice & upload a sample video to quickly create realistic lipsync videos. Discover the ease of text-to-video AI.

Search any document with GPT4

Add your PDF, Word, HTML or Text docs, train our AI on them with OpenAI embeddings & vector search and then process results with a GPT3 script. This workflow is perfect for anything NOT in ChatGPT: 250-page compliance PDFs, training manuals, your diary, etc.

AI Animation Generator

Create AI-generated Animation without relying on complex CoLab notebooks. Input your prompts + keyframes and bring your ideas to life using the animation capabilities of Gooey & Stable Diffusion's Deforum. For more help on how to use the tool visit https://www.help.gooey.ai/learn-animation

Compare AI Image Generators

Create multiple AI photos from one prompt using Stable Diffusion (1.5 -> 2.1, Open/Midjourney), DallE, and other models. Find out which AI Image generator works best for your text prompt on comparing OpenAI, Stability.AI etc.