Having conversations leveraging existing communication tools can be a challenge for deafs. While so many live transcription and text-to-speech tools are available in the market, they do not cater to the needs of the deaf and hard of hearing and aren't designed with an accessibility focus.
We are building this focusing on the problem deaf people are facing in India by building prototypes, testing it with the deaf community, and iterating based on the feedback.
Often it could take a lot of time for the deaf person to read transcriptions and respond to them depending on various factors from person to person. SUNVA AI intelligently simplifies and provides various filters on the transcriptions to minimize the time taken to read the transcription and switch gaze between the screen and the person while communicating.
SUNVA AI provides a text-to-speech + LLM layer features that help the deaf person having speech irregularities to type less and respond more. For example, if the deaf person types "I coffee", SUNVA AI will convert it to "I would like to have a coffee" and speak it out.
Sunva frontend is a single-screen UI where you can read the intelligently simplified transcriptions and write your response, which will be spoken back in the selected speech. Our idea is to build this app, take it to our deaf community volunteers, and based on their feedback, iterate the UI to make it even better.
The first version of the POC will have the following features along with speaker diarization and text to speech.
- Text (transcript) simplification using LLM
- Text highlighting of important and critical keywords (transcript) using LLM
- Intelligently apply simplification/highlighting to the transcriptions and provide easy summarizations.
Based on the user feedback from our focus groups, we will refine and add more features to the POC.
Please go through the architecture diagram to understand how SUNVA AI works. Please raise your questions in the issues section if you have any.
- Clone the repository
- Install the dependencies
poetry install
- Install Redis and start the server in the background
redis-server &
- Create a .env file inside ops dir and add the following variables
DATABASE_URL=YOURPOSTGRES_URL
GROQ_API_KEY=your_api_key
CLAUDE_API_KEY=your_api_key
OPENAI_API_KEY=your_api_key
BASE_MODEL=model_name (eg: Claude 3 Sonnet)
SPEECH_BASE_MODEL=model_name (eg: Whisper Large)
TTS_BASE_MODEL=model_name (eg: coqui-tacotron2)
JWT_SECRET_KEY=your_secret_key
- Run the POC
uvicorn main:app --reload
- Open the browser and go to
http://localhost:8000/
- Run this to enter virtual env
poetry shell
- Install the dependencies
poetry install
There are few ways you can contribute to SUNVA AI.
- By providing feedback on the POC
- By raising issues in the issues section
- By contributing to the codebase based on the issues
- Join the SUNVA AI team by filling the p+ai volunteer form and select the SUNVA AI project.