Hello, World

Inspiration

Language barriers are a common challenge faced by millions of people when trying to learn English or communicate in new environments. Difficulty in expressing oneself fluently often leads to feelings of isolation, missed opportunities, and lack of confidence — whether in social settings or professional contexts.

We were inspired to create a tool that helps bridge these language gaps in real time, fostering inclusion rather than isolation. In addition to casual conversation practice, we wanted to build something that also helps users prepare for high-stakes situations like job interviews, elevator pitches, and networking events, where communication skills can make all the difference.

What it does

Hello, World is an AR-based English language learning platform with several key features:

• Talk with other Bitmoji avatars in an AR environment using AI-generated scenarios and dialogue.
• Get real-time feedback on your responses and English capability.
• Create any scenario you want simply by describing it — we generate it dynamically.
• Take quizzes that test your ability to choose the next best course of action in social and professional scenarios.
• Practice interviews, elevator pitches, or everyday conversations, making it a versatile tool not just for language learners but also for career preparation.
• Convenience — all of this can be done wherever you are, using AR glasses for a fully immersive experience.

How we built it

We approached the build in two main parts: backend development and AR integration.

Backend

• FastAPI was used for setting up lightweight RESTful endpoints.
• MongoDB served as our database, providing flexibility for handling diverse user inputs.
• Gemini 2.0-Flash was integrated to handle robust natural language processing, with a large token capacity for complex conversations.
• Deployment was managed through Render and Azure to ensure scalability and reliability.

Frontend

• Lens Studio was chosen because we had access to Snap Spectacles, enabling a unique real-time AR experience.
• Real-time speech recognition on the Spectacles allowed us to quickly respond to user input.
• Much of the AR work involved adapting and modifying existing Lens Studio components based on advice from Snap employees, rather than building entirely from scratch.

Challenges we ran into

• Lens Studio was not intuitive or beginner-friendly, which made AR development challenging.
• Building AR applications in real-time on a new hardware platform (Spectacles) required a lot of debugging and rapid problem-solving.
• Integrating real-time speech feedback while maintaining low latency was technically demanding.
• Managing multiple backend services (FastAPI, MongoDB, Render, Azure) added complexity, especially under the tight hackathon timeline.

Accomplishments that we're proud of

• Successfully integrated Snap Spectacles with real-time AI interaction, something few teams attempted.
• Built a full-stack AR + AI application in under 36 hours.
• Created a dynamic scenario generator that adapts to any user input.
• Provided real-time language feedback to users in an engaging, accessible way.

What we learned

• AR development requires a very different design mindset compared to traditional mobile/web apps.
• Real-time applications must handle network latency gracefully — designing around slight delays is crucial.
• Building with scalable backends like FastAPI and MongoDB is key when dealing with dynamic and unpredictable data.
• Working with generative AI in a real-time context forces you to think about interaction design differently.

What's next for Hello, World

• Expanding support beyond English to include other major languages (Spanish, Mandarin, etc.).
• Allowing users to save and review their conversations to track progress over time.
• Adding achievement badges to gamify the learning experience.
• Improving real-time feedback with more advanced pronunciation coaching using audio analysis.
• Enhancing the experience with more realistic avatars and body language feedback during conversations.
• Expanding training scenarios for job interviews, business meetings, networking events, and other professional situations, helping users prepare for real-world communication beyond just casual conversation.

Built With

• azure
• fastapi
• gemini
• javascript
• lens-studio
• mongodb
• python
• render
• typescript