cardioAR

• 

• 

• 

• 

• 

• 

  scan to interact

• 

Inspiration

Over a month, our team lead found great interest in the YouTube channel Our History. The channel has a subchannel called, Autopsy, where doctors talk about why some famous stars suddenly die. He noticed a trend in about 99% of the autopsy reports that most of them die from cardiac arrest or respiratory issues due to unhealthy lifestyles like excessive smoking and snouting of illegal drugs, overworking, lack of sleep, and alcohol abuse.

This was a popular trend in the 90's for stars; ''The moment you become a star 🌟 , you need to have a cigar 🚬'' . This was part of their costume because by then, the contemporaries of that period did not know about the adverse effects of smoking.

It was the above coupled with his passion for practical education that inspired him to get us to work on CardioAR to help inform/educate people about the heart and lungs as these organs are vital to our well-being and can cause serious health issues when not cared for.

What it does

CardioAr allows learners to interact immersively with 3D assets, gifs, and User Interfaces of the human heart. Learners get to immerse themselves in this AR scene hence giving them a different experience from the the traditional way of learning.

How we built it

We built CardioAR using Adobe Aero for arranging/placing the 3D models and other assets in the scene and creating the interactions for the User interfaces that were created in Figma.

We initially designed the User interfaces in Figma, downloaded some free 3D models from Sketchfab, some gifs from makegif.com and free audio files from Wikipedia, and and some images from pexels.com and maxhealthcare.com.

We then created a folder for each asset and uploaded them into the Adobe Aero environment. Then we placed the assets in the desired locations in the scene. After a series of tries and errors with the placements of the assets, we later used the interactivity feature in Aero to add interactions to the assets; hiding and showing some of the assets on start and on click events; playing a heartbeat sound when the heart is shown beating, invoke the gifs of the heart and respiratory system to play when the corresponding annotation is clicked and display the UI describing the chambers of the heart when the annotation on the heart is/ are clicked.

Also, to inform the user that an annotation is clickable and will display a piece of hidden information, we make the hotspots/annotations bounce infinitely so it can draw the attention of users.

Moreover, users can display all the assets in the scene when the laptop hotspot is clicked and can hide them by clicking on Zara( the animated female character in the scene).

Challenges we ran into

We ran into file size limitations and found the desired 3D model, audio file, and gifs for our project. From reading about AR, we noticed that projects created in Adobe Aero work best under 50 MB ( file size), giving users a great experience. So we had to do some compression and also limit the number of 3D models in the scene. Live testing of the project was very frustrating since the orientation wasn't what we wanted after several attempts with different devices. But we look forward in the future to create a better experience.

Also, looking at the short time left for us, we had to work with a few annotations of the heart just so we could work on other parts of the project.

To add to the above, sometimes position and the placement of the assets in the scene is daunting because the position might be wrong if you don't look at the various angles to place your assets. For example, just looking at the front view of an asstes might appear to be closer but upon looking at it from a side view, top view, etc, you'd notice it could be way off the desired location.

Accomplishments that we're proud of

We are proud that we finished this project within two days and are well-positioned about what we've produced.

What we learned

We learned about teamwork and shared responsibilities in that, since two of the team members were writing exams, and given the two days we learned about this hackathon, we were able to deliver on our assigned roles in the hackathon.

What's next for CardioAR

1. Add interactive quizzes via Google Forms 2 . Build more AR science projects 3 . Build a React App to house projects 4 . Add more annotations to parts and add more 3D models
2. Migrate to other tools like Unity, modelviewer, and three.js
3. Change the name to match a universal name so that we can have varied 3D subjects.
4. Disable the tap-on card to toggle visible feature to using a close icon on each card to toggle visibility.

Moreover, to scale, if Aero crashes when the file size is large, we will look forward to working on a similar project in Unity or Google's model viewer or leveraging the power of three.js. we can then build a react app to host multiple Ar projects we create based on biological parts.

Successful testlink Live test-failureslink

System Requirements* Testing with Andriod-Beta: link Testing with IOS Device:link System Requirments: https://helpx.adobe.com/aero/system-requirements-macos-windows.html

Resources: GitHub:link Test Project: link Realistic Human Heart:link Lung and heart image:link Heartbeat sound:link Heartbeat gif:link Respirationlink

Slide: link

Built With

• adobe-aero
• figma
• makegif
• maxhealthcare
• pexels
• sketchfab