HawkEyes

For the working demo, please start from 00:47 → 🔗 Click Here (watch in 1.5x)

We’re submitting for the following track: Confluent Challenge

RedDivider

Inspiration 💡

Every year, school shootings take hundreds of innocent lives in the United States alone. Behind every number are innocent children and teachers. Every second matters. Most harm happens before help arrives. Yet these events are captured on camera. In 2025 alone, nearly 240 such incidents on school grounds were recorded, resulting in 204 casualties.

Even with these numbers, our systems still watch but don’t act. They show what happened, never what’s about to happen. This needs to change, and now, we finally have the chance to act before lives are lost.

That question became HawkEyes 🦅

transparent-divider

What is HawkEyes? 🦅

HawkEyes is real-time intelligence for physical spaces. It turns ordinary CCTV networks into a single, coherent system that understands movement, behavior, and intent across an entire facility.

Instead of isolated video feeds watched by tired human eyes, HawkEyes builds a living 3D view of a site and continuously reasons over what’s happening inside it. It doesn’t just monitor. It correlates, learns, and flags risks early — when prevention is still possible!

$Demo\_App$

transparent-divider

What it does 🤔

HawkEyes acts as an intelligent AI layer on top of existing surveillance infrastructure. After onboarding, users create a precise 3D site plan—buildings, rooms, pathways, and camera placements with yaw, pitch, and field of view. This turns flat video into spatial understanding.

Live RTSP feeds are streamed through Confluent Cloud, where every frame is processed in real-time. Object detection runs continuously, identifying people, vehicles, and critical objects. The real power comes from cross-camera correlation: when a person moves across multiple cameras, HawkEyes tracks a single, continuous 3D path instead of fragmented clips.

All detections, trajectories, and behavioral signals flow into Google Cloud (Vertex AI) for analysis and learning. Over time, the system learns what “normal” looks like for your environment and highlights what doesn’t — early!

Capability	What it enables	Preview
3D-Site Intelligence	Cameras understand where they are and what they see in the real world.
Cross-Camera Tracking	A single person is tracked across the entire facility without blind spots.
Behavioral Anomaly Detection	Flags unusual pacing, loitering, restricted access, or abandoned objects.
Context-Rich Alerts	Alerts include who, where, how long, and full movement history — Powered by Confluent Cloud and Google Cloud's Vertex AI.
Unified Command Dashboard	One live view instead of dozens of disconnected screens.

💡 “Cameras don’t get tired. AI doesn’t look away!”

transparent-divider

User Flow 🎢

Onboard & Sign In: Create an account and access the web dashboard.
Build the Site: Use the visual wizard to model buildings, rooms, and camera placements in 3D.
Connect Cameras: Add RTSP feeds; HawkEyes handles ingestion and synchronization.
Go Live: Streams flow through Confluent Cloud and are analyzed frame by frame with the help of Gemini-2.5-flash
Detect & Correlate: Objects and people are tracked across cameras in real time.
Get Alerts: Security teams receive prioritized alerts with full spatial context.
Review & Learn: Historical data improves future detection and threat scoring.

transparent-divider

How we built it ⚙️

The web application is built with Next.js focusing on clarity and speed under pressure.

At the core is a streaming-first architecture. Confluent Cloud acts as the backbone, ingesting raw video frames and structured metadata at scale. Apache Flink processes streams with sub-second latency, enriching frames with spatial and temporal context.

On the intelligence side, Google Cloud Vertex AI powers detection and behavior analysis. Vision models handle object detection on every frame, while Gemini (used as a judge model in the loop) helps classify risk levels and anomalies. Structured outputs are stored in BigQuery for fast analytics and Google Cloud Storage for long-term history.

$System\_Architecture\_Figure$

Everything is designed to scale — from a single building to multiple campuses over a mesh network, without changing how operators interact with the system! ⚡

transparent-divider

Challenges we ran into 😤

Synchronizing multiple camera feeds while maintaining real-time performance was non-trivial. Video frames are heavy, noisy, and continuous. Designing stream schemas that worked well across ingestion, processing, and analytics took several iterations. Another challenge was avoiding alert fatigue. Flagging everything is easy, but flagging only what matters required careful tuning and contextual scoring! We focused on prioritization, not volume!

Accomplishments that we’re proud of ✨

We built a system that actually runs end-to-end, not just a concept demo! Live streams flow through Confluent Cloud, detections are processed in real time, and alerts surface instantly!

The 3D correlation across cameras—turning fragmented views into a single movement path—is something we’re especially proud of. It changes how security teams perceive a space.

What we learned 🙌

Real-time systems punish shortcuts. Small design decisions in data flow, latency, and UI clarity compound quickly at scale. We also learned that in security, less noise is more trust. A quiet, accurate alert is worth far more than a loud, constant one.

Team

What’s next for HawkEyes 🚀

We further plan to deepen behavioral models, improve predictive scoring, and expand multi-site support. Longer-term, we see HawkEyes becoming a standard intelligence layer for schools, campuses, and public infrastructure — quietly working in the background so that nothing happens.

License — Apache-2.0

NOTE: API credentials have been revoked. Use your own credentials to run locally.