Pantea

Providing medical equity and disaster relief through live streaming.

Inspiration

With the recent increase in the prevalence of telemedicine and scare with epidemics such as the coronavirus-2019, we were inspired to create a mode of information dissemination that could aid people in areas where medical help is either out of reach or overextended. Pantea is the culmination of our group’s mixed skillsets, all brought together to appease a common passion for healthcare, equity, and social good. We decided to explore a new mode of group and community medical help; while interfacing access to medical advice and demonstrations, the app allows for real-time data updates of disease progression and natural disaster spread and thus gives the world a better idea of the situation being faced by people in affected areas and a better plan to center aid efforts. This tool also acts to give accessibility to people who are unable to access emergency medical resources due to rural location, overburdened medical systems, natural disasters, financial hindrances, etc.

What it does

Pantea makes medical information, outside support, and healthcare consultation available to all. People can connect easily to medical resources for the purpose of learning local disaster information, consulting real-time with experts, and joining relief or support groups. These resources include live video streams of verified medical professionals addressing specific issues as well as a video bank full of past categorized live streams along with other published medical help videos (EMT training videos, disaster survival videos, illness care videos, etc.) and articles published by reliable sources. While the user is collecting help for their medical emergency, in the background the app is collecting anonymous geolocation data which is then used to update a map of the situation in order to help track disease outbreaks, wildfires, etc. This up-to-date pinpointed geolocation information is an invaluable resource to infectious disease experts, governments, environmentalists, and aid organizations wishing to track the extent of damage or provide support/aid. The livestreaming and chat features helps to further connect livestreaming medical professionals to people who may need help or have questions, while also acting as a community connection feature which can connect people with similar issues and in similar areas to possibly act as aids or connections during disastrous times. Pantea also works to automate the diagnosis process itself, giving the user the option to either take a diagnosis questionnaire, search up specific medical conditions, or input images of a medical issue to connect the user to remote medical resources. The decision tree for self-described medical condition diagnosis asks specific symptomatic questions and allows an algorithmic identification of a probable disease. Meanwhile, user image input utilizes computer vision by running input data through a convolutional neural network trained on multiple visual ailment datasets.

How we built it

There are many working parts that came together for our final product. Our backend is written in Node.js and is the backbone for our project. After much research, we scripted a questionnaire within Node.js which allows for diagnosis of certain diseases, injuries, and natural disasters. For the span of this hackathon we decided to focus on the relevant coronavirus-2019 disease and earthquake aid. We implemented Mux’s API for live video streaming, which was perfectly suited to the task because it provides a lightweight, instantaneous way to host/share video (rather than uploading to a large site like YouTube, which is also banned in certain countries like China). We implemented a video streaming service which allows for doctors to stream medical advice to victims of natural disasters and disease outbreaks without having to be physically there. We also made this extremely simple and straightforward by implementing QR code stream starting for medical professionals. We created the pipeline for collecting patient GIS and time diagnosis data for research on outbreaks, and impoverishment. We also allow for users to view this data to track outbreaks for themselves without having to wait for media response. One of the other working parts of the project that came together was an image classification program using alwaysAI’s API along with our own Tensorflow convolutional neural network. This was used for classification of broken/fractured arms and legs for physical/visual diagnosis assistance along with the results of our questionnaire.

Challenges we ran into

Undertaking such a complicated task, it is unsurprising that we have faced many challenges throughout the build of this application. One of our biggest issues came with seamlessly integrating the database with the backend. We had to use some creative workarounds to deal with the infrastructure already in place within the code. Some issues also arose with the data collection and processing for our machine learning aspect due to the difficulty of having to generate a very specific dataset for which few examples readily exist even through web scraping. This was accomplished through intensive web searches and data preprocessing, as well as a specifically built convolutional neural network for the ML model. As in all hackathons, the biggest challenge comes with the crunch for time, but through our organizational and collaborative skills we were able to implement the majority of elements that were initially hoped for.

Accomplishments that we're proud of

While developing this project, our greatest accomplishment was being able to have a finished product with multiple levels of functionality and application. The application has multiple facets working within it: live video streaming, video archival tools, ML assistance, and geographic tracking. Interfacing so many different APIs and other software platforms all within node.js in such a limited amount of time is an incredible feat that our entire team is proud of.

We are also pleased with the accessible and aesthetic user interface, adding to both the practical and personal appeals of our product.

What we learned

After many hours of work toiling over whiteboards and computers, each of us has learned a lot about product viability. All in all, though, we learned a lot more about project collaboration with a group of wildly different skill sets and how to best organize time to maximize productivity. We learned how to laugh at our faults and remain levelheaded when plans are no longer feasible.

What's next for Pantea

With this project in particular, the largest resource needed is data: data for disease diagnosis, data for injury identification, data for treatment options. A significantly larger dataset would be needed for truly effective machine learning identification of broken/fractured limbs, skin diseases, and other visible ailments. Having a very large resource database for treatment options, advice, and medical help is also instrumental in the effectiveness of our product, so that will be expanded through computational methods as well as in general over time due to an archival of live streams. We would also love to add in resources and help group options for those with mental illnesses who need assistance. Eventually, we hope for the possibility of a donation interface within the app that allows users or disaster-trackers to donate to different causes around the globe which are being tracked and whose scope is easily visible through the app.