MediScript

• 

  logo

• 

  list of patients

• 

  patient profile

• 

  voice to text in progress

• 

  voice to text completed

• 

  wireframing

Inspiration

Unleashing Untapped Technological Potential: With the rapid development of technology, it’s puzzling why nearly all documentation processes are still manual or highly decentralized. Processes such as transcribing are often outsourced to international talent, translation is often on a doctor-to-doctor basis, and summarization of patient records is often manual as well. While new AI solutions are able to aid processes like summarization, the healthcare industry remains in a precocious stage, juggling issues like security and privacy as well as the highly crucial nature of yielding data as accurately as possible.

Filling the Gap: We noticed a significant gap in the current solutions available: they were either highly labor-dependent or current technological documentation websites/applications are too generalized and/or work independently. Therefore, current solutions fall short, resulting in issues like inaccuracy and inefficiency. Thus, the need for a comprehensive and targeted solution that solves the pain points of documentation woes of doctors is one we are seeking to address.

Informed by User Research: Our journey was propelled by insights from extensive user research. Findings illuminated the frustrations and challenges faced by healthcare professionals when dealing with existing solutions, be it using documentation applications independently, or manual documentation modalities. We saw firsthand the pressing need for a more streamlined and user-friendly healthcare app.

What it does

Mediscript is a consolidated patient-centered platform for the documentation and organization of patient data.

With voice-to-text transcription and translation from almost every language, we are able to reliably replace this labor-reliant aspect of patient documentation. The image-to-text / scanning feature allows for the consolidation of past patient records. The summarization features allow for a more streamlined organization of patient data in a more concise, consistent and digestible manner.

The organization of data by the patient for new information (eg. transcripts / scanned documents) to be easily built on and compared/contrasted to that of existing records (eg. previous visits, external screenings, etc).

Apart from greater efficiency and accuracy of documentation, we also allow for greater access, mainly through our offline transcription feature, which enables real-time accurate documentation even during most rural house visits and ad-hoc medical checks.

Having a targeted documentation service built around the healthcare professionals themselves means a more targeted workflow, privacy and security at the forefront of our model, and a close-to-zero room for error policy.

How we built it

Jess had the idea to create an app that could help medical professionals perform difficult or time-consuming tasks with little cost. After working with RNs to get the initial product specifications, we came up with the idea of MediScript, an app that can help healthcare professionals more thoroughly and quickly understand their patient’s needs through transcription, translation, and summarization.

We began by drafting out ideas on a whiteboard. Extensively. We diagramed the flow of the app, our ideal UI, and the specific aspects we wanted to focus on when developing. Through Figma and Miro, we drafted out samples and sketches so we had a good idea of what the app would look like. We thoroughly considered our logo, colors, and design, so that the app is as intuitive as possible to use, with subtle text and smooth animation.

Using Whisper and a custom LLAMA model, we fine-tuned our software specifically for medical use cases. Not only can it understand patients speaking in different languages, with different intonations or volumes, or different speech patterns, but it’s also capable of filtering out most background noise so it works even in extremely noisy situations.

Lastly, by combining general datasets with more medical-specific datasets, we curated a system that is able to understand both, to make for a smoother and more appropriate experience in a medical or hospital setting.

Challenges we ran into

We encountered several challenges during this journey, many of which involved stepping into unfamiliar territory. There were many “firsts” for us: transitioning from software development to design and delving into development areas with limited prior experience both yielded their own learning curves.

Additionally, we grappled with the tough but essential task of prioritization within our constrained timeframe, such as deciding whether to implement all features at once or concentrating on delivering one exceptional feature for the initial phase.

Distilling the "why" we built this and the "so what" was not an easy feat as well, in terms of being able to articulate the grandeur of our vision with precise language that can resonate with potential users.

Accomplishments that we're proud of

We are happy with our successful implementation of all the main features of our application, as well as the establishment of robust branding (cohesive brand identity, brand voice, and UI/UX design). Market validation is successful as well, with healthcare professionals giving detailed feedback on how they can see themselves highly integrating our application into their workflow. Personally, we are grateful to have expanded our knowledge in previously unexplored realms and not been daunted by the many “firsts” we had. Time management was great as well, in building at a pace we were comfortable with yet excited to create.

What we learned

While there is a pull towards generalization of technology or applications (eg. general summarization AI technology), there is not always a one-sized-fits-all solution. Just like how the niche characteristics of the healthcare industry require more specialized solutions, we also realised that our application has its limitations in being a one-sized-fits-all. Therefore, we used a more fluid design that accounted for many edge cases, included many error preventions and room for forgiveness in our UIUX.

The importance of user-centric design that caters to our specific target audience was also a big takeaway for us. When we did user walkthroughs of our application, there were many edge cases and possible error user flows that were not intuitive to us. Hearing from the perspective of a healthcare professional and imagining ourselves in their shoes through user persona journeys allowed for more intricate UIUX design discoveries and decisions.

Hard skills: learning to code in a different language (TypeScript), and dabbling into UIUX / design for the first time.

Soft skills: learning when to prioritize aspects of the app, learning the importance of keeping in the loop about every part of the project (eg. working on the UIUX, it was helpful knowing the basics of the programming languages we are using in order to know what are possible limitations, what would be a more intuitive development workflow, on top of purely distilling the app-building approach from the perspective of a designer).

What's next for MediScript

At this stage, MediScript’s main features are up and running. We aim to tighten our application and start with Beta testing in January 2024. We are currently speaking with a number of small-scale healthcare companies to do our preliminary user research and testing, the rationale being that these small-scale companies lack the budget to outsource documentation services like scribing to foreign labor or to adopt existing technologies to automate such processes.

Our next phase includes testing and partnerships with major hospitals in San Francisco (e.g. UCSF Medical Center, San Francisco VA Medical, John Muir Health-Walnut Creek Medical Center) as well as minor private healthcare providers or more niche providers (e.g. Sutter Care, VITAS Healthcare and Maitri-Compassionate Care) in the hospice space. We plan to integrate our system with the current EHS system then, where we will continue testing, surveying, and improving our system until it is ready to go live.

Our timeline is to make MediScript live in January 2025 with 3 main partners, spanning across public and private organizations.

Let MediScript document, let healthcare professionals focus on the people that need them the most.

(More about our pricing model: Our system allows healthcare workers to transcribe, translate, and summarise as much as they would like for just $30 per user per month, online or offline, securely. Existing systems cost $15 an hour, and doctors lose 1.5 hours after each visit due to documentation. Hospice or out-of-office doctors also have visits left undocumented due to last-minute visits, network issues, etc.)

Built With

• gguf
• llama
• node.js
• python
• pytorch
• react-native
• tamagui
• whisper