Please Read Instructions Carefully Before Use

Inspiration

Over the last century, eye health problems from eye strain, like myopia and presbyopia, have increased by an order of magnitude. Currently, half of all adults suffer from vision problems that continue to worsen with age. And the latest technologies, like VR/ Apple Vision Pros, Davinci Robots, or even cell phones, can cause nausea, headaches, and vision health problems.

It’s clear that the modern world utilizes optical technology in a way that suits the object that is being viewed, not the person. What if we made a device that uses the same technology that makes phone cameras and telescopes operate to improve the eye health of individuals and professionals alike?

What it does

Please Read Instructions Carefully Before Use is an automatic varifocal lens system with intelligent target tracking and audiovisual feedback. The user speaks commands, and the glasses will quickly track and focus on the specified objects, adjusting zoom and improving the visual experience by automatically focusing according to the eye prescription of the user. A heads-up projector displays the tracked objects, and could be extended to display identification information (such as blood vessels for surgeons, wiring for electricians), not only improving but also augmenting the human experience.

How we built it

Please Read Instructions is built with a Raspberry Pi Zero 2 base connected to a DRV8833 motor driver, two OLED screens, and a Pi Pico with a microphone serving as the audio preprocessor. The entire thing has a couple of hours of runtime off a single 9V battery. A single lead screw adjusts the focal length to provide unparalleled optical response.

On the software side, she uses Google Gemini to track objects to focus and Laplacian filters coupled to a basic closed-loop control system to gradient-descent the camera until the region of interest is in focus. Her voice generation is done through ElevenLabs to provide audio feedback.

Challenges we ran into

Miniaturization is hard! Getting this much hardware crammed inside a relatively constrained volume proved to be a pretty interesting packaging optimization problem.

Integration is hard! While each system worked individually, actually tying everything together to work cohesively was a challenge. For example, the microphone itself worked, but the communication between the microphone and the hardware layer needed to go through a hardware layer, which ended up being highly unreliable, and so we needed to adapt. To solve this, we offloaded the audio extraction and processing to a secondary device via WiFi.

Optics are hard! Our original plan, which would have allowed for a more unconstrained combination of magnification and focus, involved three lenses. This not only mechanically makes the movement much more difficult to build, but it turns out that the equations to solve it are nonlinear! We had an interesting tangent with nonlinear optimization solvers before we decided that a two-lens system would be simpler and more robust.

Accomplishments that we're proud of

One of our goals was to make something with a dash of creativity, rather than the purely technical and practical builds that our team is used to. Keyswitches stolen from a mechanical keyboard serve as the three side controls. The side OLED displays status information and also animations of cats. On a little bit of a deeper level, we wanted to create an object to explore human-computer interfaces; by having a large language model literally optically distort the view of the world, we invite people to confront how mediated their perception already is, and to question where the interface ends, and interpretation begins. We hope you’ll think it looks as wonderful and whimsical as we envisioned it to be.

What we learned

Attempting to resin print and spin coat lenses is really hard
Web browsers will not allow usage of the SpeechRecognitionAPI without a non-self-signed https certificate, but this can be bypassed by enabling unsafely-treat-insecure-origin-as-secure on the target device

What's next for Please Read Instructions Carefully Before Use

Miniaturization
Shape-changing lenses for more reliable operation and versatility
LCD selectively activates lenses to create a varifocal system with no moving parts

Justifications for Tracks

Healthcare: reduction in eye strain, demonstration replacement for bifocal/progressive lenses for presbyopia. uses in dental and surgical work, where being able to resolve different details, possibly without hand contact, is important.
Entertainment: Explores human computer interaction by literally shaping the way you see the world, unlike other AR/VR devices. Not only can it improve existing AR/VR experiences, reduce health issues, and give more people the opportunity to experience AR/VR, but it also improves quality and human capacity for near-activity.