Replae

Inspiration

Replae started with a simple moment we’ve all felt:

You open an old notebook — messy lecture notes, a forgotten goal, a journal entry from when life felt different. You read it… but you don’t really hear it. Your thoughts stay trapped on the page: scattered, overwhelming, easy to forget.

We asked:

“What if your mind could speak back to you?”

That question became the soul of Replae.

We wanted a tool that could take handwritten thoughts — the most personal, human kind of thinking — and bring them to life again. Not by rewriting them, but by giving them a voice: clear, structured, and emotionally resonant.

Replae is for students drowning in notes, for presenters who lose their voice, for people with dyslexia or low vision, and for anyone who just wants their ideas to finally make sense out loud.

What it does

Replae transforms handwritten thoughts into clear, organized audio you can listen to — like giving your mind a voice.

With a single photo, Replae can:

Extract handwriting

You upload an image of:

lecture notes

journal pages

to-do lists

childhood writing

planning pages

notes from someone you love

Replae reads it using advanced vision OCR.

Turn it into clean, usable text

Replae processes the handwriting to:

preserve punctuation

fix spacing

clean noise

remove distortions

Your messy handwriting becomes readable digital text.

Enhance your thoughts (Enhanced Mode)

Using Claude, Replae can instantly transform your handwritten text into:

structured study notes

clear summaries

prioritized to-do lists

presentation scripts

outlines

action steps

clarity-optimized rewrites

You decide what you want.

Preserve the original meaning (Raw Mode)

If you want no AI rewriting:

Replae reads the text exactly as written

perfect for journals, memories, and notes from someone important

It keeps the emotion and authenticity intact.

Convert everything to expressive audio

Replae plays the text out loud in:

A clean synthetic voice

Or a personal uploaded voice

Use cases:

studying while walking

reliving memories

practicing presentations

accessibility for dyslexia or low vision

speaking when your voice is gone

Save and revisit your entries

Each upload becomes an entry you can:

replay

reorganize

talk with in Enhanced Mode

turn into additional audio

keep as a diary or script

In short:

Replae takes a picture of your handwriting and turns it into:

organized text

structured insights

clear summaries

spoken audio

and emotional memories brought back to life

It gives your thoughts clarity. It gives your ideas structure. It gives your memories a voice.

How we built it

Backend

Flask (Python) — handles image uploads, OCR requests, AI routing, audio generation, and multi-mode logic.

Frontend

React + TypeScript + Vite — fast, modular UI with real-time state updates and smooth transitions.

OCR (Optical Character Recognition)

OpenAI GPT-4.1 Vision API — accurately extracts text from handwritten notes, journals, photos, and pages.

AI Assistant

Anthropic Claude API — powers Enhanced Mode with:

summaries

outlines

presentation scripts

study notes

action items

clarity rewrites

Text-to-Speech

Fish Audio API — generates expressive, natural-sounding audio from extracted and enhanced text.

Styling

Tailwind CSS — custom gradients, glassmorphism effects, animations, responsive layout.

Icons

Lucide React — consistent and modern UI icon system.

Challenges we ran into

Integrating Multiple API Keys We worked with several external services (OpenAI Vision, Anthropic Claude, and Fish Audio), each with different authentication flows, rate limits, and response structures. Coordinating these systems and managing environment variables across local and deployed environments required careful debugging.

Merge Conflicts During Rapid Development Because we were building the frontend, backend, OCR pipeline, and AI logic in parallel, we frequently encountered merge conflicts. Rapid iteration meant we had to repeatedly resolve code overlaps, restructure files, and keep our branches aligned without breaking the build.

AI Hallucinations and Unreliable Output Enhanced Mode needed consistent summaries, outlines, and scripts. Early versions produced hallucinations, formatting drift, or missing structure. We had to design guardrails, structured prompting, and post-processing to maintain reliability.

OCR Accuracy With Handwriting Handwritten input varied widely. Achieving clear extraction required experimentation with rotation correction, contrast enhancement, thresholding, sharpening, and punctuation restoration. Even then, cleaning and normalizing the text became a significant challenge.

Accomplishments that we're proud of

Built a fully working pipeline from handwritten image → OCR → AI enhancement → audio output

Integrated OpenAI Vision, Anthropic Claude, and Fish Audio into a unified workflow

Created high-accuracy preprocessing for handwritten OCR (rotation fix, denoise, thresholding, contrast enhancement)

Designed a clean, modern, mobile-responsive UI with multiple modes

Implemented a reliable multi-mode AI assistant (Raw, Enhanced, Study, Presentation, Clarity)

Achieved stable audio replay using user voice samples

Delivered an emotional, impactful user experience that feels personal and useful

Finished the entire application end-to-end within the hackathon time

What we learned

How to preprocess images for better OCR accuracy (contrast boosting, rotation correction, thresholding, noise reduction)

How to integrate multiple AI services (OpenAI Vision, Anthropic Claude, Fish Audio) into a single pipeline

How to design structurable prompts that produce consistent summaries, scripts, and outlines

How to handle API rate limits, retries, and error states gracefully

How to build a responsive, mobile-ready React frontend with real-time text + audio features

How to collaborate quickly while resolving merge conflicts and syncing large changes

How to design a product around emotional impact, not just technical capability

What's next for Replae

• Audio and video uploads

Allowing users to upload voice notes, lectures, and video clips so Replae can extract text, organize it, and convert it into high-quality audio summaries or scripts.

• Full mobile app version

A native iOS/Android experience so users can capture notes, images, and audio directly from their phone and replay insights instantly.

• More voice options — synthetic and fictional

Support for a wider variety of AI voices, including synthetic professional tones, character-style voices, and more personal custom voices.

• Podcast mode

A feature that turns a series of uploaded notes into a structured audio diary or personalized “daily podcast,” organizing your thoughts into episodes.

• Better audio/voice storage system

More efficient, scalable, and secure storage of user voice uploads and generated audio, with better caching, organization, and retrieval.