SymptoAI

Homepage
User Info
Loading Screen
Results Page

Inspiration

Our inspiration came from one of our teammates. He had recently injured his ankle during soccer practice. The next available appointment for him was in 3 days. He didn’t know what to do, or how to ease the pain. When he shared this story with us, we realized the problem with current Healthcare: The results arrive very slow. Then after hearing about this hackathon’s prompt, and how it coincided with our problems, we decided we wanted to make an app that would accurately deliver results within 2 minutes, with minimal information from the user. It can also give the user some information on their medical problem, and people they could contact.

What it does

Firstly, the user will login through their google account. Then they will receive some free credits to start with, 10000 credits, which they receive from their profile. After that, they can enter in some information in a box, which will pop up a lot of things to enter, but it’s very quick. After the user info has been entered, it will take some time to process and deliver results. The results will show 5 possible problems the user could have. As they read through, they can see which one corresponds the most according to their symptoms, and contact doctors according to it.

How we built it

We built our project by creating a Next.js frontend that’s wired up to a Flask server that uses OpenAI’s APIs and a vector DB that we set up using pinecone, which would match the results from the OpenAI API, to the metadata from the pinecone index. We also use OpenAI, to process the user info, and give us results in the format of a dictionary. We also had a MySQL database provisioned on PlanetScale for storing user data that was connected to using Prisma from the Next.js Node.js Vercel Lambda functions. For handling payments, we used Stripe as it handled a lot of the international problems, taxing, and several other regulations and laws that have to be dealt with when it comes to payments. We also scraped Yelp’s pages to find the best match for a doctor to cure the predicted disease. For authentication, we made use of Clerk’s APIs so we don’t have to host our own authentication routes.

Challenges we ran into

One challenge we ran into was the accuracy of our results. Initially, we used a Vector DB to upsert pages scraped from Mayo Clinic, but we found that to be inaccurate compared to raw OpenAI chat completion output. Payments was a concept that was new to us, so we had to go through several barriers to do it right and make sure the application is completely secure, despite both the server and client being open sourced. Lambda function timeouts was another problem we faced. A lot of the AI processing we did on the backend was heavy, so we faced timeouts when running the completions. To handle this, we used the lambda function to initialize a connection that was managed on our Redis database, and used a monolith server to verify the connection and process it. This allowed us to bypass the function timeouts.

Accomplishments that we're proud of

We were also really proud to learn about how embeddings work and their usefulness in a lot of situations. Without vector DB’s we might have not known how to start the project, and it also helped us solve the problem with Yelp’s irregular categorization. Together, we put this together into an entire full stack application that uses AI, payments, database, and authentication technology and full-scale. We will use vector DBs for sure in future projects due to the speed in both upserting and querying them.

What we learned

Payments – Payments are difficult to handle and keep secure, so we had to go through several lengths to make sure this was the case. Vector DBs & embeddings - This was a completely new topic for us, and we had to explore the OpenAI and pinecone documentation in great length, to have a good level of understanding on vector DBs and embedding

What's next for SymptoAI

We really wanted to figure out how to integrate our results with healthcare systems, for greater interoperability, but couldn’t do so, due to the challenges there are while accessing confidential patient information. In the near future, we would like to work with healthcare facilities, to integrate patient data into our results, but also keeping patient data confidentiality existing. All in all, we are excited to see where the project could be heading!