echopilot

echopilot.png

Inspiration

Human connection is the most fundamental part of who we are. Every relationship, friendship, and collaboration begins with a conversation — yet conversations are also where many of us struggle. Many people struggle to remember details, maintain flow, or think of what to say next. For neurodivergent individuals or those with social anxiety, this challenge can be even greater. We wanted to build something that augments human memory and empathy in real time — making conversations smoother, more inclusive, and more powerful.

What It Does

Our app acts as a real-time companion during conversations:

Listens to audio in real-time from your phone and generates a structured conversation memory of topics and chains-of-thought.
Organizes by topic — each conversation is broken into themes with summaries and transcripts, enabling you to look back on what the conversation was about.
Generates recommendations:
- Continue threads: “Ask more about their trip to Japan.”
- Pivot gracefully: “They mentioned soccer earlier, you could bring that up.”
- Recall personal context: “Last week they said they were working on a research project. Ask if you could contribute.”

We're calling it augmented memory + augmented relationships.

How We Built It

Frontend: React Native + Expo for a cross-platform mobile experience.
Audio capture: Expo AV for real-time, byte-level audio streaming.
Streaming: WebSockets to send audio chunks directly to our Flask-based backend in milliseconds.
AI Layer: Powered by Vertex AI, we orchestrated multiple Gemini-2.5-Flash's to parse transcripts into topic stacks and generate natural suggestions per topic.

Challenges We Ran Into

Real-time audio streaming: Avoiding delays and glitches when sending audio chunks through WebSockets.
Expo limitations: Some native modules (like WebRTC) required workarounds because Expo Go's sandbox environment doesn’t support them.
Balancing augmentation vs. privacy: Designing the system so it feels supportive rather than controlling — conversations are deeply personal.

What We Learned

Human augmentation isn’t just about physical enhancements but having a companion to your everyday experiences – and struggles.
Low-level byte streaming between iOS, Desktop, and browser environments.

What’s Next

Integrating WebRTC and kernel-level microphone access for lower-latency audio streaming.
Adding personalized coaching insights: “You often change topics quickly — try digging deeper into one area.”
Experimenting with AR glasses or earbuds as a device-free interface.
Expanding into cross-cultural support, suggesting conversation starters tailored to someone’s background.

Why It Matters

Conversations are the foundation of relationships, opportunities, and community.
By augmenting memory and empathy, we can give people a tool that makes communication less intimidating, more meaningful, and more human.