OpenAdapt.AI is an open source Generative Process Automation platform that transforms desktop automation through machine learning. Record human demonstrations, train vision-language models, and deploy agents that adapt to any software environment.

OpenAdapt.AI leverages advanced machine learning models to:

• Observe and Learn: Automatically learns from user interactions to generate automation scripts
• Automate Intelligently: Adapts to software changes, varying workflows, and complex environments
• Ensure Privacy: Built-in data protection with support for AWS Rekognition, Microsoft Presidio, and Private-ai.com

• Generative Process Automation: Automates tasks using generative AI models that learn from demonstrations
• Cross-Platform Compatibility: Works with browser apps (Chrome) and desktop applications
• Audio Narration Transcription: Converts spoken instructions into automation scripts
• Flexible Model Providers: Supports OpenAI, Anthropic, Google (online) and Ollama, vLLM (offline)
• Open Source: Developed transparently and collaboratively by the community
• Privacy by Design: Robust data handling mechanisms to protect user data

OpenAdapt.AI is built as a modular ecosystem of specialized packages:

                    +-----------------+
                    |    openadapt    |  <-- Unified CLI and meta-package
                    +-----------------+
                             |
              +--------------+---------------+
              |              |               |
              v              v               v
    +----------------+ +-----------+ +----------------+
    |openadapt-capture| |openadapt-| |openadapt-evals |
    |   (recording)   | |    ml    | |  (benchmarks)  |
    +----------------+ | (training)| +----------------+
                       +-----------+
                             |
              +--------------+---------------+
              |              |               |
              v              v               v
    +----------------+ +----------------+ +----------------+
    |openadapt-viewer| |openadapt-      | |openadapt-      |
    |(visualization) | |   grounding    | |   retrieval    |
    +----------------+ | (perception)   | | (demo search)  |
                       +----------------+ +----------------+

# Install with pip
pip install openadapt

# Or use uv (recommended)
uv tool install openadapt

# Record a demonstration
openadapt capture start --name my-task
# ... perform the task ...
openadapt capture stop

# Train a model
openadapt train --capture ./captures/my-task

# Evaluate on benchmarks
openadapt eval run --benchmark waa-mock --tasks 20

Each package can be installed independently:

pip install openadapt           # Core CLI and unified interface
pip install openadapt-ml        # ML training engine
pip install openadapt-capture   # Screen recording
pip install openadapt-grounding # UI element localization
pip install openadapt-evals     # Benchmark evaluation
pip install openadapt-retrieval # Demo similarity search

See each package's README for detailed documentation.

We welcome contributions from everyone. Whether you are a developer, data scientist, tester, or just passionate about automation, there is a place for you here.

• Contributing Guide
• Code of Conduct

Join our Community to discuss new ideas, report bugs, or seek help.

Areas of interest:

• Frontend development (React, visualization)
• Machine learning (model training, evaluation)
• Desktop integration (Windows, macOS, Linux)
• Benchmark development and evaluation
• Documentation and tutorials

Looking to implement OpenAdapt.AI in your organization? We offer expert consulting services for deployment, customization, and optimization.

Contact us at sales@openadapt.ai.

OpenAdapt.AI is made possible by the support of our contributors and partners. If you are interested in sponsoring development, check out our GitHub Sponsors page.

For general questions, feedback, or partnership inquiries: contact@openadapt.ai

OpenAdapt.AI is currently in alpha. While functional, some features are still under development. Use with caution in production environments.