Blackjack Dealer Robot

Inspiration

We wanted to explore how a physical robot could bring a classic card game to life through voice interaction and computer vision. By combining teleoperation, real‑time policy learning, and speech commands, we aimed to create an engaging demo of robotics and AI working together under tight time constraints.

What It Does

• Voice‑Driven Gameplay: Prompts players for “hit me” or “I’m done” and deals cards accordingly.
  
• Automated Dealing: Follows standard Blackjack rules to distribute cards to each player and itself.
  
• Computer Vision: Detects card values and table positions to guide the robot’s arm.
  
• Teleoperation Training: Learns a dealing policy via reinforcement learning in a makeshift stand.

How We Built It

• Robot Platform: Hugging Face’s SO‑100 leRobot controlled over serial (via /dev/ttyACM1).
  
• Speech Interface: Python script with a lightweight ASR engine for “hit me” / “I’m done” commands.
  
• Computer Vision: OpenCV routines to recognize playing cards and localize player positions.
  
• Policy Training: Reinforcement‑learning loop run on the robot, accelerated by an improvised cardboard stand.
  
• Fallback Components: Swapped in salvaged motors from a secondary robot when originals failed.

Challenges We Ran Into

• Serial Communication Errors: Read failures on /dev/ttyACM1 until we re‑flashed the motor controllers.
  
• Faulty Motors: One unit blinked red and refused to move; two additional motors overheated and needed replacement—twice.
  
• Physical Stand Constraints: A 3D‑printed rig would have taken six hours, so we improvised a cardboard mount at 5 AM.
  
• Time Pressure: Multiple hardware swaps and wiring adjustments between midnight and the demo deadline required rapid iteration.

Accomplishments That We’re Proud Of

• Working Policy by 6 AM: Despite setbacks, the robot could autonomously deal cards and respond to voice commands.
  
• Robust Teleoperation: Seamless motor swaps and torque recalibrations under extreme time pressure.
  
• End‑to‑End Demo: From player prompts to final dealer reveal, the system ran reliably at the hackathon showcase.

What We Learned

• Rapid Iteration Matters: Quick hardware swaps and software fixes kept us moving forward.
  
• Creative Problem‑Solving: Cardboard stands and motor salvage taught us to adapt when ideal tools weren’t available.
  
• Integration Complexity: Blending voice, vision, and robotics in one demo highlights the importance of modular design and thorough testing.

What’s Next for Blackjack Dealer

• Stable Stand Design: Produce a lightweight aluminum or 3D‑printed rig for consistent policy training.
  
• Improved Motor Mounts: Redesign fittings to prevent overheating and ensure reliable torque transmission.
  
• Advanced NLP: Expand voice commands to include “double down,” “split,” and natural‑language clarifications.
  
• Cloud‑Backed Vision: Offload card recognition to a server for higher accuracy and multi‑camera support.

Built With

• huggingface