-
Live Run / Demo
-
Device
-
Device Functionality: [Sensory Trigger] -> [Input] -> [Output]
Inspiration
Every year, millions of infants die from diseases such as Birth Asphyxia, Kawasaki disease, Sudden Infant Death Syndrome (SIDS), etc. This could be prevented via a responsive system to monitor and care for the baby during emergencies while keeping the parents more medically involved, and saving important data akin to a dashcam.
What it does
It is a hardware device that monitors a baby's vitals such as body temperature and heart pulse. It uses time-series analysis and predictive modeling to raise alarms if the baby might reach an emergency state and warn the parents. It then interacts with the parents via speech to instruct them with first aid, while also providing the LLM with the sensor data as context for better treatment.
How we built it
We used an Arduino to interface with the sensors, and a Raspberry Pi as the master controller. We initially planned to use the InterSystems Integrated ML Cloud for our predictive modeling, but the cloud did not support this feature yet, so we wrote our own signal processing model. Once the alarms are raised, the parents interact via speech, and it is translated to text using the Google Cloud speech-to-text API, and interacts with OpenAI's ChatGPT API with the sensor data as context. We do all the processing on the hardware itself to reduce reliance on mobile devices and speed up processing for critical events such as this.
Challenges we ran into
We did not have enough time to source a USB Microphone to present real-time interaction, so we instead ran with pre-recorded talks. The InterSystems Cloud also could not support our requirements, so we had to switch to a custom processor at the last minute.
Accomplishments that we're proud of
We worked on a very serious real-world problem and learned how to integrate cutting-edge AI/ML APIs with the hardware, especially as beginner hackers.
What's next for PediBeat
Making the device more modular and compact. Then we want to transition from purely infant-related emergencies to a device to tackle any general emergency in households.
Log in or sign up for Devpost to join the conversation.