TheLearningCompany

Watches you learn, takes actions, and holds you accountable.

Comment 7

Inspiration

We've all had the experience of reading an article or watching a lecture and feeling like we understood it, only to blank when someone asks us about it later. Reading isn't learning. Existing tools track what you consume, but nothing watches how you engage with material or holds you accountable for actually retaining it.

What it does

TheLearningCompany provides a distraction-free browser environment where an AI companion watches your study session in real-time. As you browse, it extracts key concepts and builds a timelog of your learning. The AI can also take actions within your browser to assist you.

When you end your session, TheLearningCompany quizzes you on what you studied, exposing gaps between what you read and what you actually understood. Those gaps become flashcards that can sync with study tools like Quizlet or Anki for spaced repetition, so you truly remember what you learned over time. You can also visualize your session as a knowledge graph showing how concepts connect to each other.

How we built it

  • Isolated browser: Docker container running Chromium + Xvfb + VNC, streamed via NoVNC
  • Screen analysis: Chrome DevTools Protocol captures frames that feed into Overshoot real-time vision API extracts concepts
  • Custom Overshoot SDK: We forked and extended the Overshoot JS SDK to support screen sharing via MediaStream, a capability that wasn't available out of the box
  • Dual backend: Python (FastAPI) for the AI agent via browser-use, Node.js for CDP frame streaming
  • Quiz generation: OpenRouter API with Claude/Gemini for question generation and answer evaluation
  • Spaced repetition: Flashcard export integrates with Anki for long-term retention
  • Token compression: Integrated The Token Company's bear-1 model to compress LLM prompts before API calls, reducing token usage while preserving semantic meaning. Built dual implementations in both Python (using tokenc SDK) and JavaScript (direct API) with configurable aggressiveness levels (0.0-1.0) and graceful fallbacks
  • Frontend: Vanilla JS + Vite with Apple-inspired liquid glass UI

Challenges we ran into

The Overshoot SDK didn't support screen sharing by default, so we forked the SDK and implemented MediaStream support ourselves. We had to build a custom pipeline where the CDP screenshot capture into Node.js WebSocket into Canvas into MediaStream into Overshoot SDK. Getting cookies and session affinity working through the proxy layer took hours of debugging.

Accomplishments that we're proud of

The AI doesn't just passively watch, it can actually click, type, and navigate within your study browser. The end-to-end flow works where you study something, get quizzed on it, see your gaps, and export them as flashcards for spaced repetition. We also built a polished UI that feels clean (in our opinion) despite being a hackathon project under time constraints.

What we learned

Real-time vision APIs, CDP frame streaming, WebSocket architecture at scale, forking and extending third-party SDKs under time pressure, and that debugging cookie routing through reverse proxies is painful...

Safety & Privacy

The study browser runs in an isolated Docker container so your normal desktop and browsing activity remain completely private. The AI only sees what's inside the study session. Users can pause observation at any time, and no data is stored beyond the session unless explicitly exported.

What's next for TheLearningCompany

  • Multi user learning profiles that track growth over time
  • Deployment and server handling
  • Best use of The Token Company: Tested 200+ compression algorithms as well as integrated their bear-1 compression model across our full stack (Python + JavaScript) to optimize LLM token usage, reducing API costs while maintaining prompt quality

Built With

+ 12 more
Share this project:

Updates