ThinkingCap

OLED Display
Full Cap
Full Cap Overhead

Inspiration

We were inspired by Rayban Meta Glasses and Jarvis from Iron Man (or similarly siri and alexa).

What it does

Voice activated AI powered hat, that will display its response on an OLED screen mounted to the rim of the cap. It is also able to take pictures via a connected webcam on the side of the hat, at the user's request.

How we built it

We used an open source voice to text model (VOSK), and ran it locally on the Arduino Uno Q. Upon hearing the keyphrase "Hey Jarvis", the model will record the user's following inquiry, and will use an OpenAI API to formulate a succinct response, displayed on the onboard OLED. The OpenAI agent will also be able to take photos with the camera if the user's question asks for it, and analyze the photo.

Challenges we ran into

We ran into numerous issues with the Arduino Uno Q. Being a new generation board, there was little documentation to go off of, and took a lot of testing and debugging to work through.

A misnamed .ino file, which would not run unless its name matched the folder's
Issues with the Router Bridge library, unable to convert special characters from python to C++

Accomplishments that we're proud of

We made a pretty cool device overall, the automatic photo tool was a lot of fun to use. It was also cool to work with a brand new generation board, that bridged Python and C++ in a very interesting way, and feel we made the most of what we were given.

What we learned

-Set smaller reachable goals. It is difficult to debug after taking large leaps without testing. -Modularize more, test components separately beforehand -Be more careful asking LLMs for coding assistance, as there was little documentation on the Uno Q, and the models often returned outdated info for older boards. -Learned more about using Git and working on remote systems (ssh)