ThirdEye

Inspiration

The median car on U.S. roads today was built in 2013, and millions were manufactured even earlier, with around 85 million cars on U.S. roads being built prior to 2010. While vehicle safety has improved significantly in recent years, most older cars still lack modern driver-assistance features that help drivers avoid crashes in the first place. As a result, vehicles without systems like forward collision warnings are involved in more frequent rear-end and urban crashes, particularly in dense traffic environments where hazards emerge quickly and unpredictably.

This gap matters beyond vehicles alone. In the U.S., pedestrian fatalities exceeded 7,500 deaths in 2022, highlighting how vulnerable road users are often the first to be affected when drivers lack sufficient situational awareness. Driver distraction further compounds the issue, contributing to roughly 12% of reported crashes.

At the same time, modern smartphones now have the computational power to run real-time computer vision and AI, capabilities that simply did not exist when these cars were built.

Safety should not be an exclusive commodity. ThirdEye delivers modern safety to the cars people are already driving.

What it does

ThirdEye runs entirely on a mounted smartphone, requiring no new hardware, no vehicle modifications, and no expensive add-ons. Once mounted, it provides real-time audio warnings for potential road hazards using on-device computer vision.

ThirdEye continuously tracks vehicles, pedestrians, and cyclists in the driving scene, identifying emerging risks outside the driver’s immediate focus or peripheral awareness. By analyzing motion, proximity, and relative speed, it recognizes when situations are developing rapidly and issues timely, realistic audio alerts powered by ElevenLabs, allowing drivers to stay informed without ever taking their eyes off the road.

In parallel, ThirdEye monitors driver attention using the front-facing camera with Presage Technologies, enabling alerts to adapt based not only on what is happening on the road, but also on the driver’s focus and readiness to respond.

How we built it

Our team built ThirdEye by first breaking down the core idea into various subcomponents that could be developed in parallel; for instance, the logic for the road-side and driver-side video analysis was split up, and pieced together for the app itself, which allowed us to make progress quickly.

Challenges we ran into

Issuing warnings early enough to be useful

Detecting hazards is only valuable if alerts are delivered with enough lead time for a driver to react. Simply reacting to what appears in the current frame often results in warnings that come too late. We addressed this for our project by developing a lightweight predictive approach that analyzes object motion across past frames, using relative movement and momentum to anticipate how situations are likely to evolve rather than reacting after the fact.

Avoiding alert overload without missing critical events

Road environments are noisy, with many objects entering and leaving the scene in rapid succession. A naive alert system would constantly speak, overwhelming the driver and reducing trust in the system. To solve this, we implemented a rate-limiting mechanism that suppresses repeated or low-value alerts while still allowing genuinely new or escalating hazards to break through, ensuring warnings remain meaningful and actionable.

Accomplishments that we're proud of

We’re proud of creating an iOS app and implementing on-device inference across almost all of the system.

What we learned

Making an app for iOS

We made an app for iOS using Swift for the first time; to keep latency low, we also decided to run as much as possible locally, which meant we utilized Apple features such as CoreML for running the CV model.

The reason behind YOLO's speed:

While modern smartphones are powerful, they are still limited compared to larger computers. For an application where low latency is critical, we needed an approach that could run fast enough entirely on-device. YOLO fits this well, as it detects objects in a single forward pass through a convolutional neural network, making it significantly faster than architectures that require multiple passes over an image.

Heart rate tracking with Presage

Using Presage Technologies on the driver-side camera view, we learned it was possible to estimate indicators such as heart rate fairly accurately from video alone. When combined with driver gaze analysis, this allowed us to issue alerts to distracted or potentially drowsy drivers.

What's next for ThirdEye

More advanced prediction is next; we built a base version of prediction for ThirdEye to add an element of anticipation to car and object movements, allowing it to caution drivers as events are about to unfold.

ThirdEye demonstrates how modern perception systems can extend safety benefits to millions of vehicles already on the road, without requiring new hardware or costly upgrades.