DriverCo

• 

  logo

• 

  banner

🧠 Inspiration

We’ve all had those long drives where you feel your focus slipping. Maybe your eyelids start to feel heavy, or you catch yourself yawning more often. Unfortunately, moments like these are a real danger on the road—drowsy driving causes countless accidents every year.

We thought: what if your car could not only notice that you’re getting tired, but also talk to you, check in, and help you stay alert—like a real co-driver who cares?

That’s how DriverCo was born. It’s not just a detector. It’s a smart, conversational co-pilot that’s always looking out for you.

🚗 What It Does

DriverCo uses real-time computer vision and AI to watch for signs of driver fatigue. It monitors both:

• Eye behavior (long blinks, closed eyes)
• Mouth movement (frequent yawning)

When it sees that you're getting drowsy, it steps in—initiating a voice conversation powered by Google Gemini. It might say, “Hey, you look tired. Want to crack a window?” or suggest you take a break. You can respond naturally, and it talks back, just like a smart assistant.

All of this is done hands-free using speech-to-text and text-to-speech—no buttons, no screens, just voice.

🛠️ How We Built It

We didn’t just use a pre-trained model—we went further. After an extensive search, we found a publicly available dataset with labeled facial fatigue indicators (eyes and mouth states). We fine-tuned a YOLOv8 model on this dataset, training it for 50 full epochs. The training paid off—the model adapted well to subtle fatigue cues and gave us strong accuracy in detecting both eye closure and yawning.

On top of that:

• Google Gemini handles the real-time conversation with the driver.
• Edge-TTS and speech-to-text enable natural, spoken interaction.
• Everything runs in a loop: detection triggers dialogue, dialogue keeps the driver alert.

The whole system runs in real-time, with minimal latency and intuitive, voice-first communication.

🚧 Challenges

• Finding the right dataset for our specific use case was hard. We knew we needed one with clear, annotated images for fatigue signs—and it took a lot of digging.
• Fine-tuning YOLOv8 was time-intensive but worth it. Training for 50 epochs helped us improve accuracy without overfitting.
• Bridging GenAI and Computer Vision—we had some experience with NLP and generative models before, but computer vision was new territory. Making the two work together smoothly in real time was one of our biggest challenges.

🌱 What We Learned

• Training your own model matters. Custom fine-tuning gave us much better results than using a generic YOLOv8 model.
• Fatigue detection is multi-dimensional—eye data alone isn’t enough; yawning was a critical addition.
• Voice UX makes tech feel human—Gemini helped us build not just an assistant, but a companion.
• Most importantly, we learned how to bring different AI domains together. We were more comfortable with NLP and GenAI, but this was our first time working with computer vision. Figuring out how to make them talk to each other, literally and technically, was a huge milestone for us—and we’re proud we pulled it off.

🚀 What’s Next

• Porting the system to a Raspberry Pi or Jetson Nano so it can run directly in cars.
• Expanding support for more languages and voice tones.
• Combining vision with GPS to recommend rest areas or gas stations when fatigue is detected.
• Exploring bio-signal input (like heart rate) for more comprehensive fatigue monitoring.

✨ Final Thoughts

DriverCo is a small step toward a big goal: making roads safer through smart, empathetic technology. It’s a co-driver that doesn’t just beep at you—it talks to you, encourages you, and keeps you focused when it matters most.

We’re proud of what we’ve built—and even prouder of what we learned along the way.

Built With

• colab
• gemini
• opencv
• python
• whisper
• yolov8