Delta

Delta is an autonomous, self-evolving desktop AI that writes its own code to expand capabilities in real time and manages your digital life through a sleek command-center interface.

Comment

Inspiration

We were inspired by the idea of moving beyond static AI assistants toward systems that can actively extend their own capabilities. Most AI tools today are limited to what their developers originally shipped. We wanted to explore what happens if an agent can recognize its own limitations, create new tools for itself, and evolve over time based on user needs. Delta is our attempt to prototype a constrained, self-improving agent that behaves more like a growing software system than a fixed chatbot.

What it does

Delta is an autonomous software agent that can perform tasks, recognize when it lacks a required capability, and then create new extensions (plugins) for itself to solve the problem. When a user requests something Delta cannot do (e.g., checking crypto prices), Delta plans the solution, writes a new extension, tests it, loads it into its runtime, and then executes it to fulfill the request.

Delta also supports:

  • A plugin system where new tools can be dynamically added at runtime.
  • A plugin store that allows sharing extensions across different Delta instances.
  • Visual perception, giving Delta the ability to “see” the user’s screen to assist with GUI-based tasks.
  • Real-time interaction via WebSockets.
  • Persistent memory and extensions stored in SQLite.

How we built it

Delta is built as a modular, agentic system:

  • Python for the core runtime, agent logic, and extension framework.
  • FastAPI for the HTTP and WebSocket server that powers real-time communication.
  • Google Gemini API for reasoning, planning, and code generation when Delta needs to create new tools.
  • HTML/CSS/JavaScript for the web-based dashboard and chat interface.
  • WebSockets for low-latency, bidirectional communication between the frontend and the agent.
  • SQLite for persisting memories, extensions, and metadata.
  • PowerShell/Bash scripts for system-level automation and task execution.

The system is designed so that new capabilities are loaded dynamically at runtime, without restarting the agent.

Challenges we ran into

  • Safety and control: Allowing an agent to write and execute code introduces obvious risks. We had to constrain what Delta can generate and where extensions can execute.
  • Reliability of self-generated code: Automatically generated plugins can fail in subtle ways, so we needed validation, testing hooks, and sandboxing.
  • State management: Keeping the agent’s memory, plugins, and runtime state consistent across sessions was non-trivial.
  • Tool discovery: Designing a mechanism for Delta to correctly detect when it lacks a capability and formalize that gap into a concrete tool specification.
  • Real-time UX: Ensuring the WebSocket-based interface stayed responsive while Delta was generating and testing new code in the background.

Accomplishments that we're proud of

  • A working prototype of a constrained self-improving agent that can extend its own capabilities at runtime.
  • A functional plugin architecture with dynamic loading of new tools.
  • A basic plugin store concept that allows sharing extensions between agents.
  • A real-time web interface for interacting with Delta.
  • A proof-of-concept “visual perception” pipeline that enables GUI-level assistance.

What we learned

  • Self-modifying systems are powerful but extremely easy to get wrong without strong constraints.
  • Tool abstraction and capability detection are more important than raw model intelligence.
  • Reliability and safety mechanisms (testing, sandboxing, permission boundaries) are first-class design requirements, not afterthoughts.
  • Even simple autonomous behaviors quickly introduce complex state and lifecycle management problems.
  • Building agentic systems is less about prompting and more about robust software architecture.

What's next for Delta

  • Stronger sandboxing and permission systems for self-generated extensions.
  • Better evaluation metrics so Delta can judge the quality and safety of its own code changes.
  • A more mature plugin store with versioning, ratings, and trust signals.
  • Improved visual perception for more reliable GUI automation.
  • Long-term memory and learning mechanisms so Delta can generalize improvements across sessions and users.

Built With

Share this project:

Updates