Netra AI

Interacting with querying service
Events live stream
Interacting with querying service
Audio tool
Video querying

Inspiration

Vibha (one of our teammates) was in India last summer visiting her grandparents. Someone stole something in front of her grandma's house, so she was asked through look through hours of footage to determine what exactly happened. While brainstorming, the team decided that AI could be used to address this issue and that's how Netra AI was born.

What it does

Netra AI has two main functionalities. The first one is that the user can query for events. For example, the user can ask if a person came to their house and if so, what time. The second functionality is that users can get alerts on events with detailed descriptions.

How we built it

For the frontend and backend, we used the Reflex framework. To analyze the video files we used Gemini. For the speech to text feature in the chatbot, we used DeepGram. To detect the motion and record the 5 second clip, we used OpenCV. We used ChromaDB to store the alert message produced from the processing.

Challenges we ran into

One challenge that we had was learning the Reflex framework. We wanted to display the alert messages on the website, but we were not able to do so. We also had issues with the motion detection being too sensitive, but we were able to fix it by changing the sensitivity.

Accomplishments that we're proud of

One thing we are proud of is being able to successfully implement the speech to text feature. We felt that this feature was important because it would be useful for people on the go to easily communicate with the system. Additionally, we wanted to make our system more accessible. Vibha's grandma struggles to type on the keyboard, so in this case a speech feature would be useful for people like her.

What we learned

Through this experience, we learned many technologies from the sponsors like Gemini, ChromaDB, Reflex, and DeepGram which we used in our project. We also learned how to work better as a team. We were able to make progress by delegating tasks. When we felt stuck, we swapped computers with each other to get a fresh set of eyes. This proved to be very valuable as we were able to resolve many issues this way.

What's next for Netra AI

1) Real Time Text Alerts with Detailed Descriptions of Important Events Text user that, for example “Your daughter has arrived home” or “There is a fire outside your home.” 2) More Training of the AI Model Train the AI model on video data from different countries and people. 3) Improving Security and Privacy Implement Multi-Factor Authentication and Encryption