Telebuddy

Mirrored robots
AR demonstration

Project Title: Telebuddy

Tagline: Precision soldering, without the human shake.

Inspiration

The idea was born from burnt fingers and ruined PCBs. We’ve all been there: it’s 3 AM at a hackathon, you’ve had too much caffeine, and you’re trying to solder a 0402 resistor or a fine-pitch header. Your hands are shaking just enough to bridge the pins, and suddenly your project goes up in smoke.

We realized that while software has become incredibly accessible, hardware hacking still has a massive physical barrier to entry. Fine motor skills take years to develop, and physical tremors (whether from biology, coffee, or anxiety) can make electronics frustratingly inaccessible. We wanted to build a "ctrl-z for soldering"—a system that separates the human intent from the physical execution.

What it does:

Telebuddy is a teleoperated robotic assistant that democratizes precision soldering.

Teleoperation: The user holds a 3D-printed "pen" controller. As they move the pen in the air, a robotic arm mimics the movement in real-time.

Tremor Filtering: The system intercepts the motion data and applies an EMA filter, removing high-frequency jitters (shakes) while keeping the intentional movement smooth. It turns a shaky hand into a surgeon's precision.

Virtual Practice (AR Mode): By switching modes, users can see a virtual PCB on their screen. They can practice soldering components in Augmented Reality using the same physical controller, allowing them to build muscle memory without wasting expensive components.

How we built it:

We split the project into three main pillars: The Controller, The Robot, and The Brain.

The Controller: We 3D printed a custom solder-pen grip housing 6 different rotary encoders. It reads the encoder position at each axis to determine orientation and acceleration, streaming it over UDP.

The Robot: We used a 6-DOF robotic arm modified with a soldering iron holder. The servos and stepper motors are driven by an ESP32 microcontroller.

The Brain (Software): Signal Processing: We implemented a moving average filter and a Kalman filter in Python to smooth out the noisy sensor data and eliminate human tremor. Forward Kinematics (IK): We wrote a kinematic solver to translate the rotations of the pen into specific angles for the robot's servos. AR Interface: For the practice mode, we built a web app using Ursina Engine and Unittest. It parses the information directly from the leader bot, visualizing the pen's position relative to a virtual board.

Challenges we ran into:

Material Issues: With a lack of consistent parts, we had to adapt our 3d model and firmware code to use a variety of actuators from stepper motors to servo motors.

Mapping 3D Space: Mapping the relative movement of a handheld pen to the absolute coordinate system of a robot arm was difficult due to the inverse kinematics. We carefully debugged and measured the correct dimensions.

Firmware Issues: With so many different signals being sent back and forth, we had to be very precise while coding our firmware to ensure no overlapping code or race conditions.

Accomplishments that we're proud of:

Zero-Jitter Demo: Seeing the raw data plot vs. the smoothed data plot side-by-side was incredibly satisfying.

Seamless AR Integration: Getting the physical controller to drive the virtual soldering iron in the browser and play a soldering "game" was a satisfying experience.

What we learned:

Robotics is hard: Inverse kinematics is much trickier in the real world than in simulation due to servo play and physical constraints.

Accessibility Tech: We gained a deeper appreciation for assistive technology. This project isn't just for hackathons; it has genuine potential to help people with essential tremors or motor impairments engage in electronics.

What's next for Telebuddy

Haptic Feedback: We want to add vibration motors to the pen so the user can "feel" when the robot touches the PCB.

Computer Vision Assist: Adding a camera to the robot that automatically identifies pads and "snaps" the iron tip to the center of the joint (aim assist for soldering!).

Remote Labs: Scaling this up so a student in a dorm room can control a high-end soldering station in a lab miles away, enabling remote hardware education.