Ditto

A persistent AI digital twin that learns your personality, remembers your experiences, and speaks in your voice.

Comment 1

Inspiration

Most AI tools today are generic. They respond well, but they do not know you. They forget context, lack personality, and feel interchangeable. We were inspired by a simple question:

What if an AI could actually be you?

Ditto was born from the idea of preserving identity, not just information. Humans are defined by how they think, what they experience, and how they speak. We wanted to combine psychology, memory, voice, embodiment, and blockchain to create a digital twin that feels personal, persistent, and real. Not a chatbot, but a presence.

What it does

Ditto creates a personalized AI digital twin that thinks, remembers, and speaks like you.

Users build their twin through three core inputs:

  • Personality assessment
    A structured quiz that maps cognitive style, behavior, and decision-making patterns.

  • Diary-style journaling
    Ongoing journal entries that become long-term, searchable memory for the AI.

  • Voice recording
    A short audio sample used to clone the user’s real voice.

Once created, the twin can hold natural conversations the way the user would. You or a trusted friend could ask things like:

  • “What’s been on your mind lately?”
  • “How do you usually feel about situations like this?”
  • “What would

How we built it

Ditto is a full-stack web application built around a modular AI pipeline that combines personality modeling, long-term memory, voice cloning, real-time 3D rendering, and optional blockchain verification.

On the frontend, we used Next.js with TypeScript and Tailwind CSS to build a responsive glassmorphism-style interface. The interactive 3D Ditto avatar is rendered using Three.js with React Three Fiber, allowing it to follow the cursor, react to proximity, and lip-sync in real time.

Authentication is handled through OAuth, and all user data is stored in a PostgreSQL database with strict access controls to ensure privacy. Personality quiz results and diary entries are saved securely and used to construct each user’s digital twin.

For the AI layer, diary entries are embedded into a vector database to enable Retrieval-Augmented Generation. When a user asks a question, relevant memories are retrieved and injected into the prompt alongside the user’s personality profile. A large language model then generates a response that reflects both memory and personality.

Voice cloning is handled using ElevenLabs. Users record a short audio sample directly in the browser, which is converted server-side and used to create a cloned voice model. AI responses are then spoken aloud using the user’s own voice, significantly increasing realism and emotional impact. Word-level timing data is used to drive smooth, real-time lip-sync animation on the 3D avatar.

Optionally, cryptographic hashes of the twin’s data are committed to the blockchain, creating a permanent and verifiable proof of identity without storing raw personal data on-chain.

Challenges we ran into

One of the biggest challenges was making the AI feel consistent over time rather than like a generic chatbot. This required careful structuring of personality data and memory retrieval so responses stayed true to the user.

We also had to manage latency across multiple systems including memory retrieval, language generation, voice synthesis, and rendering, all under hackathon time constraints.

Finally, integrating blockchain in a meaningful way without compromising user privacy required designing a hash-only commitment model instead of storing data directly on-chain.

Accomplishments that we’re proud of

  • Built a personality-driven AI that behaves consistently rather than relying on prompt tricks
  • Implemented real long-term memory using retrieval-based generation, not just chat history
  • Achieved voice cloning and real-time lip sync inside a web application
  • Created an embodied AI experience with an interactive 3D avatar
  • Designed a privacy-first blockchain identity system using cryptographic hashes
  • Delivered a cohesive product that combines AI, audio, 3D graphics, and Web3

What we learned

We learned that structured personality data dramatically improves how believable AI interactions feel. Memory retrieval quality matters more than raw model size, and embodiment through voice and visuals makes AI feel far more human.

We also learned that blockchain is most effective as a verification layer rather than a storage layer, and that building across disciplines requires clear boundaries between systems to move fast without breaking things.

What’s next for Ditto

Next, we want to expand Ditto’s memory system to support long-term timelines and personality evolution over time. We plan to add sharing and permission controls so users can decide who can talk to their digital twin.

Future directions also include enterprise use cases for knowledge preservation, multi-language voice support, and on-chain ownership of AI identities. Ditto is just the beginning of what persistent, personal AI can become.

Built With

Share this project:

Updates