MockTalk

• 

  MockTalk Logo

What it does

MockTalk is a bot, built to help users improve their conversational abilities. By allowing the user to talk to a live-video that constantly updates, MockTalk is able to simulate AI-powered real conversations paired with audio and visual in real-time. MockTalk also gives AI-powered suggestions based on your resume and has eye-contact tracking to ensure

How we built it

We built the project primarily through Python. We leveraged Google's Gemini API to develop a fine-tuned model that develops response to the user as well as our suggestions and improvements box. We leveraged Sync's API to generate audio/video to simulate conversation.

Challenges we ran into

Our biggest challenge along the way was the usage of our LLM model. For a large majority of the project, we were planning to use Llama 2's 7B Mistral for our interviewer's speech content, hosting the model on the Anvil supercomputers provided by the RCAC. However, we ran into many technical issues out of our control while doing so, and thus made a last minute pivot to Gemini's APIs to access an LLM without using the Anvil computers. We also ran into some issues with the GUI in Python, but we were able to resolve most of them along the way.

What's next for MockTalk

We plan to use MockTalk to help teach languages to people. When learning languages, having a partner that you can actually speak the language with, makes the difference between fluency and just memorizing random words. We also plan to improve MockTalk's interview prep abilities, by fine tuning an eye-tracking model we developed to help user's improve eye contact for interviews. We also want to fine-tune a resume analysis feature we built, where it analyzes the users' resume and makes the suggestions and conversation more personalized to the user's resume.

Built With

• gemini
• python
• sync