Iris

jarvis had friday, now you have iris. (smart glasses connecting individual hardware devices into an ecosystem of smart ones)

Comment 1

iris smart glasses

inspiration

jarvis had friday, now you have iris. (smart glasses connecting tech tgt)

honestly? we were tired of pulling out our phones for everything. checking a recipe while cooking, missing texts when your hands are full, trying to talk to someone in another language, it's all friction.

smart glasses exist but they're either $300+ or basically useless. we figured we could build something actually functional with stuff from amazon and a weekend of hacking.

so we built iris, voice-controlled glasses that show you info and let you control your smart home without touching anything.

what it does

say "hey iris" and you're in. everything is voice controlled:

  • todo list, add tasks, check them off, all hands-free
  • live translation, speaks french? see english. real-time, right in your vision
  • weather/time, glanceable info on the idle screen
  • iot control, lights, fans, sensors, all controllable by voice
  • glasses-to-glasses messaging, send texts to another pair of iris glasses

the whole flow is voice commands. "hey iris" then "connect" then "light" then "on". done. light's on. you never touched anything.

how we built it

the setup:

  • phone acts as mic + wifi hotspot (ip webcam app)
  • pi 0w is brain, speech-to-text, command parsing, api calls
  • laptop is power source, we couldn't find a suitable battery
  • arduino nano drives the oled display over usb serial
  • esp32s run all the iot devices

stack:

  • python + flask for the main app
  • speech recognition for voice-to-text
  • state machine for ui navigation
  • http apis for device control
  • openweathermap for weather
  • deepl for translation

hardware:

  • 0.96" oled (ssd1306, i2c)
  • arduino nano for glasses 1
  • esp32 for glasses 2 + all iot devices
  • pir motion sensor
  • vl53l0x tof distance sensor
  • dc motor + l298n for the fan
  • leds + resistors for the light

challenges we ran into

pi zero w was a nightmare. wifi setup kept failing, ssh was flaky. we ditched it for an arduino nano with usb serial and honestly it's way more reliable.

pi camera didn't physically fit our setup. pivoted to using the phone camera which actually worked better, higher quality and already has a mic.

half our team was in texas, half in california. we built everything with a "mock mode" so software could be tested without hardware. made integration way smoother.

128x64 pixels is not a lot of screen. every ui screen had to be designed super tight, 4 lines, ~21 chars each. no room for fluff.

voice recognition in noisy environments is rough. we picked command words that sound distinct so the system doesn't get confused.

accomplishments that we're proud of

full voice control end-to-end. wake word to device action, completely hands-free.

modular iot system, any esp32 device can join the network. adding new devices is just adding http endpoints.

mock mode saved us. the whole software stack runs without hardware. rapid iteration, easy testing.

peer-to-peer messaging actually works. two people with iris can text each other silently.

the pivot. pi zero w didn't work out, we adapted fast. arduino nano ended up being the better choice anyway.

distributed team shipping a working demo across two states. that's the real win.

what we learned

build mock mode first. parallel development is impossible without it.

simpler wins. usb serial > wifi for the main display. not everything needs to be wireless.

state machines make complex uis manageable. highly recommend.

hardware doesn't care about your plans. always have a backup.

phones are incredible sensor packages. camera, mic, wifi, all built in and high quality.

what's next for iris

  • compact camera mounted on the frame instead of phone
  • transparent oled for actual ar overlay (gotta figure out eye strain)
  • battery power so it's actually portable
  • more iot integrations, locks, thermostats, garage doors
  • offline voice recognition for lower latency
  • two-way glasses communication
  • mobile app to replace the laptop entirely

built with

python, arduino/c++, esp32, flask, speechrecognition, openweathermap api, deepl api, ssd1306 oled, pir sensor, vl53l0x tof sensor

Built With

Share this project:

Updates