Vyz

Vyz is an AI-enabled sensory adaptive system that integrates real time auditory and visual data to create calmer environments and prevent sensory overload, especially for neurodivergent people.

Comment

Inspiration

The inspiration for this project came from the 2028 LA Olympics. Many people want to root for their country surrounded by other fans. However, it can be hard for some people to fully enjoy these scenarios, as loud stadiums are prime locations for sensory overload. When feeling overstimulated, people often have to step out of the venue into a quieter location which takes away from their experience and can cause them to miss peak moments. We created Vyz so people can stay in the moment of the game, helping them overcome auditory and visual sensitivities.

What it does

Vyz is a 3 in 1 wearable system powered by AI that helps manage overstimulation. The first part are togglable sunglasses that come down over the user's eyes to block overly bright and strobing lights. The 2nd part are headphones which help cancel out loud noises and piercing high frequencies that can irritate users. The final part is ambient lights that can help soothe users and lead them through breathing exercises that can help them manage overstimulation without having to step away. All three of these solutions are integrated into the form factor of a wearable baseball hat and shoulder bag to be minimally invasive and allow people to maximize their experiences.

These 3 solutions are controlled by a web app and AI. The web app is used to have maximum user control over all the criteria. Users can control at what brightness the sunglasses come down, how strong the noise canceling is, what color lights are playing and more giving them full control of their experience. AI comes in to personalize the experience for the user. Using input from a camera and microphone, the AI can recommend breathing exercises, light patterns, and thresholds that will best help the user manage their situation and enjoy themselves to the fullest.

How we built it

The main compute behind this device is a Raspberry Pi 4B (Rpi). The Rpi is running a flask server which is used to update the thresholds for the controls of the 3 previously mentioned systems by receiving posts from the web app. The flask server also makes requests to GPT in order to create the custom recommendations that it can also use to update the thresholds. The Rpi is connected to a camera, microphone, headphones, and an Arduino which is connected to a servo to control the sunglasses and the leds. The Rpi runs python scripts to read the input from the camera and microphone and then call the correct actuators accordingly, for example when the brightness from the camera is too high, the servo is told to make the sunglasses cover the user's eyes. The actuators are controlled by Arduino instead of the Rpi because running the PWM necessary for servo control and led control interferes with the headphone audio output from the Rpi. Communication between Rpi and the Arduino is done through serial. The app was designed in Figma and then created in React using Expo.

Challenges we ran into

We ran into many issues with hardware interfacing. One major problem we ran into was performing the noise cancelation. We were originally using the Arduino as an ADC to quantize data from a contact microphone and then send it over serial to the RPi to be processed for noise cancelation. However, we found this to be extremely difficult to implement. We didn't know how to process the bits into a readable form that we could then do the signal processing to provide the sound we wanted. After some time, we decided to change our approach and instead use a USB microphone directly into the RPi. This greatly alleviated the difficulties because the RPi could then natively recognize the microphone and we could use premade libraries to process the sound. Another problem we ran into was integration. We had many different subprocesses going on with this project, with the 3 hardware solutions and the web app. It was difficult to integrate all these solutions together, especially with the limited compute power we had on the Rpi to work with. While there were many possible solutions to our problems, we had to be really specific to minimize the amount of work done on the Rpi. For example, we actuated the LEDs and the sunglass servo on the Arduino instead of the Rpi and designed our Flask server to have to make and receive the least amount of requests as possible.

Accomplishments that we're proud of

We're very proud of our entire project. We spent quite a while ideating until we found a problem we deeply resonated with that could help lots of people. Additionally, we knew that we would run into difficulties creating a hardware heavy project. However, we were able to push through and achieved the majority of our initially defined scope. We're very proud of our initial planning stages and how well we were able to follow through on the execution. There were so many pain points, but we were able to persevere thanks to our belief in our idea.

What we learned

On the technical side, we all learned a lot about both hardware and software, and specifically their integration. Many people on our team had significantly more experience with software than hardware, but there were a few that leaned the other way. We all had to push ourselves past where we were comfortable in order to integrate these hardware and software components together into one working component. We also learned and practiced the importance of communication. We were a larger team, so it was very important that we were are all on the same page on what we needed to do and what was done. Working on a hard project can be grueling and can make us forget the importance of communication. Thus, we were constantly checking on each other and collaborating. This was important to us because it helped us keep motivation high knowing some parts were working and also helped integration go smoother at the end when there was a better group understanding of the entire project.

What's next for Vyz

Electochromic Lenses

We want to implement electrochromic lenses, material that can change its transparency based on voltage to provide a more continous experience darkening and lightening experience to the user rather than the current binary sunglasses of on and off.

Better Noise Canceling

We can implement a better noise canceling system for the headphones which can cancel out sound in a better way. We could use better microphones, higher computer, and better speakers to create a more immersive experience for users while preventing themm from becoming overwhelmed.

Better Form Factor and Design

We currently need to run lots of cables and the device isn't very easy to use. We would like to make the design more seamless, so it's not as obvious that a user is using this device. Additionally, we want to make it significantly easier to use the device in general.

Share this project:

Updates