MARK I: KINETIC

MARK I: KINETIC (Not Finished) Glove when it is worn
MARK I: KINETIC (Not Finished) Glove
MARK I: KINETIC (Finished) Glove
MARK I: KINETIC (Finished) Glove when it is worn
Dataset we recorded (There's a ton!!!)
The drawing data we collected
Modal GPU where our ML will run!

Our Teams🧑‍💻

Mark I: Kinetic was conceptualized and built in 24 hours by:

Harry Nguyen – Embedded and Machine Learning Developer
Andrew Dang – Embedded and System Integration
Huy Le – CAD and Hardware Prototyping
Chi Nguyen – Operations Management and Business Analysis

Inspiration💡

Inspired by the intuitive, gesture-based technology of Iron Man Triologies, we wanted to bridge the gap between human motion and digital control. Traditional presentation clickers are clumsy and tether presenters to their laptops, so we wanted to turn the human hand itself into the ultimate interface. "Tony Stark was able to build this in a cave, with a box of scraps!"

What it does❓

Kinetic is a wearable glove that grants a presenter freedom of movement to continue their flow.

Navigation: Swipe gestures to flip slides (Left Arrow Keyboard/Right Arrow Keyboard)

Precision: Real-time mouse control via wrist tilt (MPU6050) and a magnetic "pinch-to-zoom" feature (Hall Effect sensor KY-024).

Intelligence: Uses a Bidirectional LSTM neural network to recognize hand-drawn letters in 3D space to launch specific applications (e.g., drawing a 'P' to open PowerPoint).

How we built it🔨

For Hardware: We used an ESP32-WROOM-32E as the brain, communicating over two simultaneous channels: BLE HID for instant mouse/keyboard input and WiFi TCP for streaming sensor data. Signal processing is handled by Kalman and EMA filters to ensure the mouse movement feels natural and jitter-free. We also solder our gloves to ensure no jitter and less electrical noise.

For Software: The machine learning pipeline consists of a TensorFlow LSTM model and Keras hosted on Modal (Cloud GPU) to handle complex gesture inference without lagging the local machine.

Challenges we ran into❗❗❗

Our team consists of two Electrical Engineers, one Operations Manager, and one Mechanical Engineer, where Machine Learning is going to cause a lot of problems since that isn't our focus. We want to challenge ourselves as we do Hardware Hack, with the spirit of a Hackathon that goes with a Software Hack. Another limitation we encounter is the lack of tools we have to use for the Hardware wiring process. In the end, we managed to minimize the hassle and finish our prototype.

Accomplishments that we're proud of😁

We've successfully developed a prototype of the smart gloves where the Letter Recognition Model is implemented in the real-life world. We are also proud of how we brought our childhood dream to be like Tony Stark as an Engineer/Mechanic.

What we learned📖

We also learned one thing that we thought was impossible for us to finish without a general knowledge of Machine Learning. We also gained deep experience in sensor fusion, specifically how to combine the MPU-6050 to read data and analyze an ML model effectively.

What's next for MARK I: KINETIC

There's more to our project, which could open in many branches!

Right now, what we are focused on is related to Business and Presentation. We want to focus more on the future, where the scope could reach Medical/Healthcare, Architecture/Construction, and fully support accessibility functions.