SyncroScribe

• 

  Login Page

• 

  Note-taking screen using speech-to-text AI

• 

  Example notes transcribed from speech

• 

  Confirming notes and opportunity to add additional information

• 

  Recording the verbal nurse hand-off to summarize with notes

• 

  Example hand-off transcription from speech

• 

  AI-generated summary of notes and hand-off information displayed in a dashboard

Inspiration

Through the Pittsburgh Regional Health Initiative's workshop on Saturday, we learned how many preventable deaths and injuries are caused by human error; about every one in four patients are harmed in hospitals. This motivated us to find ways in which we could use technology to reduce human error and save lives in such a vital field.

What it does

SyncroScribe leverages speech-to-text and generative AI models to ease the burden on overworked healthcare professionals. During a nurse's shift with a patient, they can easily record short snippets of information that are transcribed to text. Then, when their shift ends and they hand off their patients to the next nurse, SyncroScribe can transcribe their conversation, extracting the most important and nuanced information to be summarized and referred to later. This technology will help to reduce the loss of vital information between healthcare workers, reducing the likelihood of preventable injuries and deaths.

How we built it

We used React.js for the front-end, Whisper API from Lemonfox.ai for the speech-to-text, Meta's Llama LLM through CloudFlare's Workers API, Express.js for the back-end, and finally HTML and CSS for styling webpages.

Challenges we ran into

We spent a huge amount of time at the beginning of the Hackathon tweaking and validating our idea to ensure that it would actually be a useful technology, and (roughly) implementable in the time we had as well. Technologically speaking, the CloudFlare Worker API didn't like us calling it from the client-side through a React component, so we had to implement a back-end using Express.js to handle the API calling for CloudFlare. Our React component would call our API, which would then call the CloudFlare API and pass the response to our React component. This turned out to be for the best, though, because it made our React code a lot neater and more organized.

Accomplishments that we're proud of

We're proud firstly of our idea; we spent a lot of time making sure our idea was practical and realistic. We're also proud that we were able to execute our idea to the level that we did. Of course, with more time we could have implemented more features and made the existing ones better and easier to use.

What's next for SyncroScribe

With more time, we could fully flesh out the other aspects of the patient dashboard, like important time-based information, charts, and how previous files may relate to current conditions.

Built With

• cloudflare
• css
• express.js
• html5
• llama
• react
• whisper