Inspiration
According to the Centers for Disease Control and Prevention (CDC), TBI is a leading cause of death and disability in the United States. It is estimated that approximately 1.7 million people in the US suffer a TBI each year, with 52,000 deaths and 80,000-90,000 new cases of TBI-related long-term disability. TBI can affect people of all ages, but it is more common in children and young adults.
TBI can result in a wide range of physical, cognitive, and behavioral symptoms, and the severity of these symptoms can vary greatly depending on the extent and location of brain damage. Currently, the diagnosis of TBI is typically made through a combination of clinical assessment, imaging techniques such as CT and MRI scans, and neuropsychological testing. However, these methods can be costly and time-consuming, and may not provide a complete picture of brain function and recovery.
Objectives: The aim is to explore the potential of using open source machine learning pose estimation software for the diagnosis and measurement of brain recovery in TBI. Specifically, we will:
Evaluate the accuracy and reliability of open source machine learning pose estimation software for the diagnosis of TBI.
Determine the feasibility of using machine learning pose estimation software to measure brain recovery in TBI over time.
Investigate the potential benefits of using machine learning pose estimation software, including cost and time savings, as well as improved accuracy and precision in diagnosis and measurement of brain recovery.
How it would work
Through a pre-trained computer vision library for pose estimation - OpenPose, videos will be quickly labeled and give estimated positions of body part coordinates. This raw keypoint data can then be analyzed through academically determined scoring standards to create relevant outputs such as range of motion (ROM), joint distance, joint extension, speed of a specific motion for joint specific and complex movements.
This data can then be used to extrapolate key insights to assess the effectiveness of different training interventions. By tracking the pose of an athlete before and after a specific training program, trainers can determine whether the program had a positive impact on the athlete's movement patterns and overall performance. Furthermore, this can be used to detect deviation from standard movement to alert the the user of possible issues with motion.
Minute changes in overall motion detected through OpenPose can serve as a diagnostic for players suspected of TBI, and monitoring these changes post-injury can be used to monitor improvement and rehabilitation. On a wider scale the multi-person pose estimation capabilities of OpenPose introduce the possibility of using such detection software in training facilities of sports with high risk of TBI such as football, hockey, boxing, MMA etc.
How we built it
Wix as a website builder and in Python using OpenPose
Challenges we ran into
Minimal experience with dedicated frontend development necessitated the use of a website builder. The website builder was greatly convenient in allowing us to rapidly create the website without needing to manually code using HTML, CSS and JS. Although, It presented obvious limitations in terms of flexibility and customization beyond the available elements and styles.
We both share a common background in behavioural neuroscience and have only recently begun utilizing an interdisciplinary approach combining computation and neuroscience in pursuit of our own research
We were unable to install OpenPose on macOS which prevented us from really testing the backend code.
Accomplishments that we're proud of
This is the first hackathon for either of us and we are proud to have worked through and made a submission. We created a website that will currently serves as landing page for our project, and allows users to upload videos. We also created a separate backend that should take a video input. Partial backend that can take video input and return keypoint data in a table to further used in the analysis pipeline.
What we learned
We need to spend more time with full-stack development, backend integration tutorials and use of APIs
We plan to use cloud GPUs to implement OpenPose server side likely using Google cloud, which will require familiarity with the platform.
What's next for Open RehAIb
We will be competing in the Mind Fuel Tech Futures Challenge 2022 which begins on January 31st.
References
Badiola-Bengoa, A., & Mendez-Zorrilla, A. (2021). A systematic review of the application of camera-based human pose estimation in the field of Sport and physical exercise. Sensors, 21(18), 5996. https://doi.org/10.3390/s21185996
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017). Realtime multi-person 2D pose estimation using part affinity fields. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2017.143
Inayat, S., S. Singh, A. Ghasroddashti, Qandeel, P. Egodage, I. Q. Whishaw and M. H. Mohajerani (2019). "A toolbox for automated video analysis of rodents engaged in string-pulling: Phenotyping motor behavior of mice for sensory, whole-body and bimanual skilled hand function." bioRxiv: 2019.2012.2018.881342. https://doi.org/10.1101/2019.12.18.881342
Inayat, S. (2020). string_pulling_mouse_matlab, GitHub: String Pulling Behavioral Analytics, A Matlab-based toolbox for characterizing behavior of rodents engaged in string-pulling, https://github.com/samsoon-inayat/string_pulling_mouse_matlab, v4.0, aa7eb6c.
Log in or sign up for Devpost to join the conversation.