GeminiAtlas

• 

  GeminAltas

Inspiration

What if Pokémon GO was also a social experience that fuels your curiosity? A huge part of short-form social media today is being glued to your phone, passively scrolling, with your attention constantly fragmented – one of the main causes of the rise in learning difficulties worldwide. GeminiAtlas opposes every aspect of the "death scroll". It encourages you to go out and discover the world, to be a curious explorer, to connect with people across time and space, and to live and grow through what you learn.

What it does

GeminiAtlas is a Snap Spectacles filter that turns your everyday environment into a personalized, social, AI-powered learning experience. It’s built on several pillars: First, I align the sky with you: the Constellations Lens. This is an augmented reality Snap Lens that overlays constellations directly onto the real stars you’re seeing. With Lens Studio, I integrated 3 key controls:

1. Location: The user changes country or city. The sky recalculates in real time.
2. Compass: The Lens orients with you. You look up North, it displays the Big Dipper.
3. Voice control: The user says "Orion" or "Cassiopeia". The Lens calls up the constellation and highlights it. The result: The sky is no longer abstract. It becomes an interactive astronomy textbook. Second, I read for you: the Bookwoom Lens. Bookwoom is a Lens for Snap Spectacles. The concept is simple: the user frames a book with their hands. With Lens Studio, GeminiAtlas analyzes the cover live. In 2 seconds, I surface the info: author, summary, key themes, and similar books. The text leaves the cover and becomes a reading card you view at a glance. The result: A library becomes a living database. Third, I show you the Globe. This is the heart of GeminiAtlas. Every card you capture with Constellations or Bookwoom has a position, a subject, a memory. I organize them spatially around you. You turn your head: Noguchi in Seattle. Another turn: innovation in Cotonou. Your long-term memory becomes a landscape. The more you save, the more the experience becomes your best partner. Capture and remix! Capture any element in your environment with a simple two-hand pinch, and GeminiAtlas turns it into a short, surprising Curiosity card based on a random interest of your choice. You can then start a conversation with the AI assistant about each card to get details and learn more. We have repeatedly found that the most memorable things we learn are not the ones we study in a structured way, but the ones we discover by chance.
4. The intersection of two disciplines often leads to unexpected discoveries.
5. Our interests are the best guides and the best windows for expanding our understanding and view of the world.
6. Exploration and discovery naturally make you more curious and a better learner; it’s a virtuous circle that makes the world more exciting. Explore and share! The cards are geolocated and designed to be shared. Join your hands in front of your chest – a symbolic gesture that "discovers the world" – and sends a signal that analyzes your location and reveals a universe of cards left by you and others. Save, comment, follow, or share your own capture!
7. When you find an interesting object, discover what others have learned about it – perhaps through an interest you had never considered before.
8. When you visit a new place, explore it with those who came before you, thus sharing a moment across space and time. Relive and remember — A living, evolving memory palace in the palm of your hand, where all your saved cards are stored. The AI organizes everything spatially, by subject, relevance, and by what it learns about you. The more you save, the better the experience understands how you learn. Collect and battle! Take on the challenge and learn together by competing with friends in a Q&A game. Questions are randomly generated from both players' cards; the one with the most cards and who knows them best wins. An AI host, with offbeat humor and audio-only, reads each question aloud and reacts to your answers.

How we built it

GeminiAtlas is a Spectacles lens built with Lens Studio. All our models are processed by Snap’s Remote Services Gateway (RSG). The assistant and battle host interact over Gemini Live; image understanding and battle question generation use OpenAI Vision and GPT-4O. From an architectural standpoint, the interface consists of many subsystems coordinated by a set of global.* singletons, allowing prefabs to reach shared state without scene wiring. Each conversational agent connects to Gemini Live via RSG, streaming speech as 24 kHz PCM, with controlled transitions so only one live session is active at a time. The capture pipeline performs a two-hand crop to generate an OpenAI Vision caption. The globe-to-card transition is handled by a guided state machine that mathematically aligns the sphere and the flat card by ground footprint. Battle mode is managed by the host on SpectaclesSyncKit, with questions generated from captured cards (GPT-4o) and broadcast between the two players.

Challenges we ran into

Most of the difficulties involved coordination – between sessions, devices, and clocks. The main issue was that RSG only keeps the mostj recent session active – two simultaneous Gemini sessions drop the connection – so we had to make our voices explicitly share a single time slot. The shared microphone isj destructive and can lock up: a failed start right before launch leaves the provider "started but out of service", so we implemented a cyclical stop-restart mechanism and a monitoring system for recovery. Multiplayer was a real challenge. Because device clocks aren’t comparable, the leaderboard had to be based on the server clock. Also, Ulkit buttons misled us: their touch event handlers only fire on startup, so disabling a button at launch makes it permanently unusable.

Accomplishments that we're proud of

We’re proud that the assistant looks like a character, not just a chatbot: a strong personality, an expressive sphere that reacts to sound, real-time text captions, all running via the gateway with no on-device work. The globe-to-card navigation is exactly as we imagined: a single continuous gesture, from orbit to a street you can walk, with the transition invisible by design. Card discovery is truly magical: a plane wave in the room and the cards appear as if by magic at every touch. And we shipped a real two-player quiz, latency-tolerant, with an adaptive AI host that knows not to mock a player who’s already struggling.

What we learned

The through-line of everything we learned: decouple thought from voice and make the deterministic parts boring. Our line banks, tool declarations, and query logic are fully model-independent and easily testable; agents only own the live session. We learned to freeze what never changes and only generate what’s personal: the 37 fixed Cosmos cards send pre-written questions instantly and for free, while only user-captured cards require an API call. We learned to keep placement calculations pure and deterministic: every card scatter is seeded by the card ID, so markers never drift between frames, zooms, or sessions.

What's next for GeminiAtlas

More people, more curiosity. Currently, the globe offers Tokyo, Seattle, and Los Angeles with a universe of 37 cards and 13 initial interests. The next step is to expand the map and library to many more cities and themes, for a rich and varied discovery experience. We want to deepen the social dimension: more detailed profiles, the ability to follow people whose cards you enjoy, and themed paths through a neighborhood. We’d like to grow Battle mode beyond two players, with small-group matches and themed decks. Longer term, we want GeminiAtlas to become a living, shared atlas of human curiosity: every wall, every dish, and every street corner annotated by someone who found it interesting, waiting for the next person to pass by. In conclusion: GeminiAtlas + Snap Spectacles + Lens Studio = learning by moving, sharing, playing. The buzzer is done waiting. Ready to start?

Built With

• audio:
• gemini-live
• globe->carte
• glsl-frameworks/plateformes:-snap-lens-studio
• javascript
• khz
• langages:-typescript
• machine
• openai-gpt-4o-architecture:-global.*-singletons-pour-l?etat-partage
• openai-vision-api
• pcm
• pour
• snap-spectacles
• spectaclessynckit-services-cloud/ia:-snap-remote-services-gateway-rsg
• state
• streaming
• transitions