Inspiration
Teammate's grandma had an unfortunate accident via falling while sweeping the floor - causing her a significant amount of discomfort and pain from what we often take to be a seemingly mundane event.
What it does
The project has a back end that has real time detection on human objects for falls. Once a fall is detected a number on file - intended to be a caretaker, relative, or concerned individual is contacted via call. This is intended to be stored and be able to be accessed via a webpage front end for which the skeleton exists.
The Backend uses Python version 3.11.x with OpenCV, MediaPipe, Numby, and Twilio.
The Frontend uses ReactJS and JavaScript.
The connection between the two is done via Flask.
How we built it
The main functionality is with OpenCV, MediaPipe, Numby.
Using the users' camera as input the program determines if it has detected a fall, and will continue to take live input until exited with either 'q' or a fall detection.
If the user falls a message is displayed to notify that a fall was detected, the frame in which it was detected is captured, and a phone call is placed to the number on file via Twilio.
Challenges we ran into
- Connecting back-end and front-end, how to and implementation.
- Set up and connecting databases to the log in page which we still need to figure out.
- Figuring out how to detect falls from a side view as many body parts are hidden from that angle.
- Should we focus on heavier body part tracking that notices many tiny parts of the body or general Haar cascade (the green rectangles you sometimes see in video surveillance feeds)?
- How to deal with body parts out of frame if we're using them for evaluation of a fall.
- Dependency optimization (heavy load).
- Trying to understand how to deploy to a website instead of localhost, which we still need to figure out.
Accomplishments that we're proud of
We are all beginners, early on in university, with minor coding class instruction. That in mind, we are proud that we were all able to use Git effectively to collaborate most of all. We also had no front end knowledge prior and learned everything from scratch, much like for the back end we had no experience with Python libraries. We're proud to have been able to learn all that, and implement a working connection between the two, with a tangible working solution.
What we learned
We learned how to research packages in a short period of time and evaluate whether they would work for us or not - including short term implementation tests. We learned project management and effectively turn a high-level concept into actionable steps with deliverable pieces. We learned how to collaborate amongst people working on both different parts of the project and the same parts of the project. We learned how to adapt quickly on a time crunch if something does not work out. We learned Git and internalized the Git workflow in a group setting pretty well.
What's next for Fall Guys
What's next includes the implementation of the database connection to the log in page as well as for storing video feeds to replay for the medical account. There is also an intention of a transcription feature using LLM to transcribe conversations between the doctor and another - whether the elderly themselves or the concerned parties - especially highlighting if any serious health consequences or medication is mentioned. We also were hoping to deploy this on a website instead of localhost.
Log in or sign up for Devpost to join the conversation.