Apical

Home Page
Focus Check
Knowledge Check

Inspiration

Nowadays, with the advent of new social media apps, everything is fighting for our attention. This is an especially bad situation for students. We were inspired by this to use computer vision to help students stay on task.

What it does

You declare a start time for a task and what you are learning. Then the program will start your webcam and track your head and eye movement to determine if you are paying attention. At the end of this task, it will tell you the percent of time you paid attention and present you with quiz questions based on what topic you are learning.

How we built it

We used svelte for the frontend. We used Gemini API to generate the questions. We used Keras and linear algebra to determine your eye and face positions and determine if you are paying attention. We put this together using Flask to combine the frontend and backend. We also used Google OAuth to let a user sign in.

Challenges we ran into

We had trouble with linear algebra when figuring out the math to find out if someone is looking or not. We had trouble integrating the Keras model into the web app. Finally, we had problems with the percent calculation.

Accomplishments that we're proud of

We are proud of being able to integrate a complex backend with an aesthetically pleasing frontend despite having limited experience.

What we learned

We learned how to use Gemini API, Keras, frontend development with Svelte, Google OAuth, and how to use linear algebra to find facial position with the camera in dlib.