SkribeMonkey

Patient Dashboard
Login Screen
Register Patient Screen
Sample Patient Information

Inspiration

When discussing project ideas for this Hackathon, our teammate Rayyan voiced a plight shared by many of his friends entering the medical industry: how mentally taxing inscribing can be amidst the fast-paced Emergency Room environment. Each of his friends expressed that they wished there was a way to ease the process, as many modern software programs employed by hospitals nationwide are complex and build more stress for students aspiring to be medical professionals. We knew we could make a simple yet effective mobile app that could change how medical scribes perform their jobs. This would allow them to focus more on patients, streamlining the process for everyone involved.

What it does

Built with Flutter, Supabase, and around the IOS platform, we provide an intuitive mobile app that everyone in the industry can quickly pick up. Medical professionals can sign up, authenticate their accounts, and then add their patients to their personal dashboard. By clicking on a patient, they can see their full name, email address, gender, conditions, and an "urgency level" designated by Open-AI's GPT 3.5-turbo API. Everything fits on one screen, so users don't need to sift through files and spreadsheets to find information. On this screen, users can also upload an audio file that can be instantly transcribed by Open-AI's Whisper API, which is then summarized in the patient view. An automatic summary of an interaction with a patient further relieves the cognitive load for healthcare professionals dealing with a mass influx of patients.

How we built it

Our front end was built using Flutter, and our back end uses Supabase. We implemented a database schema in Supabase for Patients, Medical Professionals, and Patient Visit Entries. These can be queried using PostgreSQL and referenced by the front end. We also employed two APIs designed by Open-AI, Whisper and GPT 3.5 Turbo. We used Whisper to transcribe medical appointments and employed GPT 3.5-turbo to summarize this transcription for future reference and data. We also created a custom logo to accompany the project to give it an extra friendly touch.

Challenges we ran into

The first problem we ran into was that 3/4 of our team members had never used Flutter and had barely any experience with full-stack development. A lot of what we had to learn in this experience was how to work as a team to build a cohesive application. This required us to learn Git, where we immediately had to deal with a merge conflict on our first few pushes, just from the README file. The middle stages of development were smooth and caused fewer problems since our team was getting used to the project framework. Our final step was the most crucial, integrating our speech-to-text transcription. This caused us the most issues, taking up many development hours. Flutter library bugs, corrupted files, incompatible codecs, Xcode build fails, and testing small changes became our task for hours the night before submission.

Accomplishments that we're proud of

We are proud that, being a team with only one member competing in a Hackathon before, with a small project, we could push out a full-stack application with minimal experience and no boilerplate code or templates. Learning new languages, frameworks, and tools is always challenging, but our excitement for SkribeMonkey and its application to help our peers kept us going.

What we learned

We learned a lot about collaborating on a software project as a team, especially with version control and how to attack multiple fronts. Our first few merge conflicts and struggle to push to a repository branch were enough for us to ingrain these skills into our minds. As mentioned, we also learned Flutter, PostgreSQL, and design principles. Although brief, we learned some Typescript before scrapping the ideas involved.

What's next for SkribeMonkey

Throughout this experience, we had many plans for SkribeMonkey, but not all of them made it to the final submission, as many were put off to work on the transcription process. Our first edits will go towards implementing the front end for the search bar to search patients by name, as the back end is already written. Other functions we need to implement for the front end include deleting and updating users, patients, and medical entries. Additionally, we expressed interest in using a multi-class classification machine learning model to diagnose patients based on their symptoms. For now, though, we plan to show this to our peers in the medical industry and see what they think of it, potentially opening SkribeMonkey to more functionality and maturity.