F.A.V.R

Reimagining Education Through A Visual Lens(Beginner Project)

Comment 1

Inspiration

My group always struggled in school with long passages of text, often losing focus on the topic itself. We felt there must be a better way to learn in the modern age of technology. A methodology that could support our visual learning needs. The answer was F.A.V.R.

What it does

F.A.V.R. (First Aid Virtual Reality) is a state-of-the-art VR simulator that revolutionizes visual learning by providing immersive, hands-on experiences. Initially designed to teach CPR, F.A.V.R. integrates advanced technologies to ensure users master critical techniques effectively.

F.A.V.R. places users in realistic, interactive scenarios, allowing them to practice CPR in a controlled virtual environment. Using Convolutional Neural Networks (CNNs), F.A.V.R. analyzes user performance in real-time, providing immediate, personalized feedback on technique and accuracy. The VR environment simulates real-life situations, offering a more engaging and effective learning experience compared to traditional text-based methods. What makes F.A.V.R unique to any of its competitors is how it leverages VR and AI. F.A.V.R. adapts to each learner's needs and monitors the user's behavior, ensuring a comprehensive understanding of life-saving procedures.

How we built it

First, we used Blender to create all the 3D models, like the human figure and other objects needed for the simulator. These models were essential for creating a realistic environment for CPR training. Next, we faced challenges connecting our Oculus Quest headset to our laptop, but we resolved them to ensure smooth integration with the VR simulator built in Unreal Engine 5.1.

In Unreal Engine, we focused on coding with C++. This allowed us to develop the core functionality of the simulator, including how the CPR actions and interactions would work within the virtual environment. C++ in Unreal Engine provided the tools needed to fine-tune the simulation's behavior and responsiveness, crucial for medical training accuracy.

To track and analyze CPR metrics, such as compression rate and accuracy, we integrated a Python script using the Unreal API. This setup enabled us to gather data from the simulation and process it efficiently. We opted to use the Google Mediapipe API for hand tracking, which suited our project's needs without requiring extensive data resources.

Challenges we ran into

In developing our CPR simulator, we faced several key challenges. Initially, connecting our Meta Quest headset to the laptop for simulator deployment took eight hours of troubleshooting due to compatibility issues. We explored alternatives like using Android Studio to sideload builds onto the Oculus. Despite overcoming connectivity hurdles, implementing these workarounds introduced new technical issues at each step, typical of non-standard solutions.

Integrating Python scripts into Unreal Engine also posed challenges. Errors arose from incorrect object accesses within the engine, requiring meticulous debugging to ensure proper functionality. These experiences underscored the technical complexities of VR development and reinforced our problem-solving skills in navigating integration challenges effectively.

Accomplishments that we're proud of

We're proud to have developed a functional model that effectively educates users on CPR guidelines. This achievement not only validates our efforts but also sets the stage for future educational simulators. Our vision is to create accessible training tools for niche professions like doctors, astronauts, and trade schools, enabling learners to practice their skills affordably before real-world application. This accomplishment motivates us to expand our impact in education through immersive technology.

What we learned

Everything in this project was new territory for us, aside from Python. Despite the steep learning curve, mastering Blender for 3D modeling and becoming proficient in Unreal Engine stood out. We learned to integrate devices like Oculus for testing, understood game mechanics, and navigated Unreal Engine's complex structure. C++ became essential for coding within Unreal Engine, enhancing our understanding of game development and pushing our technical skills forward.

What's next for F.A.V.R

Our next goal is to transition F.A.V.R. into a startup that partners with local trade schools and doctorate programs, making a tangible impact in education. Virtual reality has the potential to revolutionize learning, providing crucial support for visual learners and personalized teaching methods. We aim to expand our reach and enhance educational experiences, ensuring that practical skills training becomes more accessible and effective than ever before.

Share this project:

Updates