Brailliant

You too can grow your brailliant mind

Comment

Brailliant: Grow Your Brailliant Mind

💡 Inspiration

Braille literacy is a fundamental right, yet a staggering number of visually impaired individuals lack access to effective, modern learning tools. Inspired by language apps like Duolingo, Renshu, and Busuu, we asked ourselves: Why can’t learning Braille be just as engaging and accessible? We set out to build a device that makes mastering Braille intuitive through active recall.

🔍 What it does

Brailliant is an interactive tutor that bridges the gap between digital learning and physical touch. The system consists of two primary hardware components integrated with an AI-driven voice interface:

  • The Braille Display: A 6-servo actuated cell that physically raises and lowers pins, allowing users to "read" braille patterns in real-time.
  • The Braille Keypad: A custom input grid where users practice active recall by typing out the patterns for letters, numbers, and punctuation. It is a 2x3 grid that perfectly mirrors the braille cell, allowing the user to feel their writing before using the save and submit buttons to check their answer.

Through ElevenLabs and Groq, Brailliant offers voice-guided lessons. Users can request specific learning plans using natural language (e.g., "I want to practice elevator numbers"), and the device responds with a customised learning plan, creating a personalised educational experience.

⚙️ Features

Curriculum

  • Active Recall Loop: Hear a prompt -> Type the pattern -> Feel your answer on the display -> Submit
  • Pattern Recognition: Feel the pattern -> Say the answer -> Hear the correct answer
  • Numbers: To identify currency value, use elevators and read signs
  • Basic Letters + Punctuation: To build the foundation of braille literacy
  • Compacted Words: Grade 2 braille words for emergency and everyday scenarios (ex: north, south, exit, danger, push, pull, etc..)
  • Keeps track of mistakes and only removes them from the mistake history once the user gets them right in a review session
  • Customised lesson plan is built based on the users needs by combining entries from our 3 modules

Keypad

  • Allows for multi-cell input by saving a current cell state when done and putting in the next one before submitting it for checking
  • Updates the braille display live with input, to allow the user to feel their answer before submitting
  • Toggles each pin state, allowing the user to undo silly mistakes

Braille Cell

  • Displays braille using a 2x3 grid of pins
  • Cycles through multi-cell braille with a 5s delay in between cells

Voice Interface

  • Fully voice controlled experience, no GUI required
  • Allows for conversational responses, making the learning experience more comfortable

🛠️ How we built it

  • The Brain: An ESP-32 microcontroller manages the physical inputs and outputs, paired with Python-based logic.
  • The Voice: We integrated the ElevenLabs API to provide empathetic vocal instructions that move beyond robotic TTS.
  • The Logic: Groq processes natural language inputs, allowing the user to control their learning journey through conversation and filtering lesson plans based on user needs.
  • The Hardware: We utilized a 6-servo design for the display and a 4x4 keypad to allow for active recall and physical typing.

🧠 Challenges we ran into

  • The Hardware: Wiring 6 servos onto a single board was a major challenge. Managing the power draw to avoid frying components was pretty hard (but we made it!)
  • Cross-Platform Integration: Integrating the physical hardware with our software stack was difficult as the systems were developed across different operating systems. This required rewriting certain portions of the code to ensure the ESP-32 and the voice APIs functioned properly together.

🏆 Accomplishments that we're proud of

We successfully moved Braille learning from a static experience to a "living" device with cheap components, allowing anyone to rebuild this. Getting the hardware to respond in perfect synchronization with the AI voice prompts was a huge win. We are especially proud of the Active Recall loop, where a user hears a prompt and physically inputs the answer, with the option to correct mistakes and feel their response before submitting.

📖 What we learned

We gained experience in hardware-software interfacing, specifically managing PWM signals for multiple servos, finding clever ways to interact with the firmware through python and integrating Real-Time AI APIs with the final project. (functional programming is super goated 🔥)

🚀 What's next for Brailliant

  • Expanded Curriculum: Moving beyond our current basic library into a full curriculum of "Compact Form" Braille and everyday words.
  • User Profiles & Memory: Implementing a GUI and memory feature to allow multiple users to save their progress and resume lessons. Currently, it only stores mistakes for a single user.
  • Smart Priority: Creating a priority list for lessons that adapts to the user's personal learning speed and difficulty areas.

Tech Stack

Hardware: ESP-32 Microcontroller, SG90 Servos, 4x4 Keypad
AI & Software: Groq (LLM), ElevenLabs (Voice API), Python, C++

Black Servo Holder and Pins: https://www.instructables.com/BrailleBot/ The rest of the CAD was done by us!

Built With

Share this project:

Updates