Automated AI Bug Fixing Pipeline - From Sentry alert to GitHub PR in seconds

Reflex is an intelligent bug-fixing service that automatically reproduces production errors in isolated environments, generates AI-powered patches using Google Gemini, validates fixes with automated testing, and creates pull requests - all without human intervention.

Reflex transforms production debugging from a manual, time-consuming process into an automated workflow:

1. 🚨 Sentry detects a crash (e.g., ZeroDivisionError in production)
2. 📡 Webhook triggers Reflex with error traces and stack info
3. 🐳 Daytona spins up an isolated sandbox (20-second cold start)
4. 🧪 Reproduces the bug by running tests in the exact environment
5. 🤖 Gemini 2.5 Flash generates a patch based on error context
6. ✅ Validates the fix by re-running tests
7. 🔄 Retries up to 3 times with feedback if initial patch fails
8. 🚀 Creates a GitHub PR with the fix automatically
9. 🧹 Cleans up - sandbox auto-destroyed

Real Example: Reflex fixed this ZeroDivisionError in 47 seconds - from Sentry alert to merged PR.

┌─────────────┐      ┌──────────────┐      ┌─────────────────┐
│   Sentry    │─────▶│   FastAPI    │─────▶│  Daytona SDK    │
│   Webhook   │      │   Server     │      │   Sandbox       │
└─────────────┘      └──────────────┘      └─────────────────┘
                            │                        │
                            │                        ▼
                            │              ┌──────────────────┐
                            │              │ Clone Repo       │
                            │              │ Run Tests (fail) │
                            │              └──────────────────┘
                            │                        │
                            ▼                        ▼
                     ┌──────────────┐      ┌──────────────────┐
                     │ Gemini 2.5   │◀─────│ Send Error Trace │
                     │ Flash API    │      │ + File Context   │
                     └──────────────┘      └──────────────────┘
                            │                        │
                            │                        ▼
                            │              ┌──────────────────┐
                            │              │ Apply Patch      │
                            │              │ (Direct FS API)  │
                            │              └──────────────────┘
                            │                        │
                            │                        ▼
                            │              ┌──────────────────┐
                            │              │ Rerun Tests      │
                            │              │ (6/6 passing ✅) │
                            │              └──────────────────┘
                            │                        │
                            ▼                        ▼
                     ┌──────────────┐      ┌──────────────────┐
                     │ GitHub API   │◀─────│ Git Commit+Push  │
                     │ Create PR    │      │ Cleanup Sandbox  │
                     └──────────────┘      └──────────────────┘

• HMAC signature verification for all Sentry webhooks
• Isolated sandboxes - each bug fix runs in a disposable container
• Auto-cleanup - sandboxes destroyed after 20 minutes
• Path guardrails - AI can only modify source code directories

• Gemini 2.5 Flash with 1M token context window
• Retry loop - up to 3 attempts with feedback on patch failures
• Context-aware - sends full repo structure and file contents
• Safety filters disabled - can read full stack traces without censorship
• Direct file modification - bypasses fragile git apply with custom diff parser

• Before/after testing - ensures fix actually resolves the issue
• File-based output capture - workaround for Daytona SDK stdout/stderr limitations
• Pytest integration - validates with existing test suites
• Rollback on failure - no PR created if tests don't pass

• Automatic PR creation with formatted descriptions
• Branch naming - reflex/fix-{issue_id}-{timestamp}
• Commit messages - extracted from error type

Live Pull Request: manuvikash/reflex-test#1

Timeline of Automated Fix:

00:00 - Sentry webhook received (ZeroDivisionError)
00:03 - Daytona sandbox created (7ebf2a8f-0797-4ed4-92b3-e879f89388d0)
00:08 - Repository cloned, tests executed (1 failed, 5 passed)
00:15 - Gemini generated patch (attempt 1 failed, attempt 2 succeeded)
00:22 - Patch applied via direct file modification
00:28 - Tests re-run (6 passed, 0 failed ✅)
00:35 - Git commit created and pushed
00:42 - GitHub PR created: "Fix: ZeroDivisionError: division by zero"
00:47 - Sandbox cleaned up

The Fix:

# Before (caused crash)
def average(numbers):
    return sum(numbers) / len(numbers)  # ZeroDivisionError on empty list

# After (AI-generated)
def average(numbers):
    if not numbers:
        raise ValueError("Cannot calculate average of empty list")
    return sum(numbers) / len(numbers)

• Python 3.12+ (tested on 3.12.0)
• Daytona Account - Sign up free
• GitHub PAT - Personal access token with repo scope
• Sentry Account - Any tier works (free tier supported)
• Google AI Studio API Key - Get one free

# Clone the repo
git clone https://github.com/manuvikash/reflex.git
cd reflex

# Install dependencies
pip install -r requirements.txt

# Set up environment
cp .env.example .env
# Edit .env with your API keys (see Configuration below)

# Create Daytona snapshot (one-time setup)
bash scripts/make_snapshot.sh

Create .env file with these keys:

# Daytona (get from https://app.daytona.io/settings/api-keys)
DAYTONA_API_KEY=dt_abc123...
DAYTONA_API_URL=https://app.daytona.io/api
DAYTONA_TARGET=us

# Sentry (get webhook secret from Internal Integration)
SENTRY_WEBHOOK_SECRET=whsec_abc123...

# GitHub (create PAT at https://github.com/settings/tokens)
GITHUB_OWNER=your-username
GITHUB_REPO=your-test-repo
GITHUB_TOKEN=ghp_abc123...

# Google Gemini (get from https://aistudio.google.com/apikey)
GOOGLE_API_KEY=AIza...
GEMINI_MODEL=gemini-2.5-flash

# Patcher mode
PATCHER_MODE=api  # Uses Gemini API (default)

# Start webhook server
python -m control.server

# Server runs on http://0.0.0.0:8000
# Webhook endpoint: POST /webhooks/sentry
# Health check: GET /health

1. Go to Sentry → Settings → Developer Settings → Internal Integrations
2. Click New Internal Integration
3. Name: Reflex
4. Webhook URL: https://your-domain.com/webhooks/sentry (use ngrok for testing)
5. Permissions: None required (webhook only)
6. Enable Issue Alerts webhook
7. Copy the Webhook Secret to .env as SENTRY_WEBHOOK_SECRET

# Run the buggy example app (triggers Sentry)
cd examples
bash run_buggy_app.sh

# This will:
# 1. Send error to Sentry
# 2. Trigger Reflex webhook
# 3. Auto-create PR with fix

# control/server.py
@app.post("/webhooks/sentry")
async def sentry_webhook(request: Request):
    # Verify HMAC signature
    # Parse Sentry payload
    # Trigger async worker

# control/worker.py
async def handle_sentry_alert():
    # Create Daytona sandbox from snapshot
    sandbox = daytona.create_sandbox(
        snapshot="reflex-ci",
        timeout_minutes=20
    )
    
    # Clone repo and run tests
    daytona.clone_repo(repo_url, branch, commit)
    test_output = daytona.run_command(
        "pytest -q > /tmp/test_output.txt 2>&1"
    )

# control/patcher.py
def generate_patch(error_trace, file_content, repo_context):
    response = gemini.generate_content(
        f"""Fix this error:
        {error_trace}
        
        File content:
        {file_content}
        
        Repo structure:
        {repo_context}
        
        Return ONLY a unified diff patch.""",
        safety_settings={
            # All categories: BLOCK_NONE
        }
    )
    return extract_diff(response.text)

# control/daytona_client.py
def apply_patch_file(patch_path):
    # Parse unified diff manually
    match = re.search(r'--- a/(.+?)\n\+\+\+ b/(.+?)\n(.*)', 
                      patch, re.DOTALL)
    
    # Extract old/new content from hunks
    old_content = extract_old_lines(hunks)
    new_content = extract_new_lines(hunks)
    
    # Direct string replacement
    file_content = read_file(filepath)
    updated = file_content.replace(old_content, new_content)
    write_file(filepath, updated)

# control/worker.py
for attempt in range(3):
    apply_patch()
    retest_output = run_tests()
    
    if "passed" in retest_output and "failed" not in retest_output:
        # Tests passed!
        create_branch()
        commit_changes()
        push_to_github()
        create_pull_request()
        break
    else:
        # Retry with feedback
        regenerate_patch(feedback=retest_output)

Problem: process.exec() returns empty stdout/stderr
Solution: File-based output redirection

# Doesn't work
output = sandbox.process.exec("pytest -q")  # Returns empty

# Works
sandbox.process.exec("pytest -q > /tmp/output.txt 2>&1")
output = sandbox.fs.download_file("/tmp/output.txt")

Problem: git apply rejects patches for trailing whitespace, line ending mismatches
Solution: Custom unified diff parser with direct file modification

# Old approach (fragile)
sandbox.process.exec(f"git apply {patch_file}")  # Fails 80% of time

# New approach (robust)
apply_patch_file(patch_path)  # Custom parser with FS API

Problem: API blocks error traces as "dangerous content"
Solution: Disable all safety filters

safety_settings = {
    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,
    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
}

Problem: Gemini references non-existent file paths (e.g., src/calculator.py when repo has calculator.py)
Solution: Send full repo structure to Gemini

repo_files = sandbox.process.exec(
    "find . -type f -name '*.py' | head -20"
)
# Include in prompt: "Available files: {repo_files}"

✅ End-to-end automation - Zero human intervention from alert to PR
✅ Production deployment - Running on real Sentry errors
✅ Intelligent retry - Feedback loop with up to 3 patch attempts
✅ Safety guardrails - Path restrictions, line limits, signature verification
✅ Sandbox isolation - Daytona ephemeral environments with auto-cleanup
✅ Test-driven validation - Only creates PR if tests pass
✅ Real-world fix - Successfully fixed ZeroDivisionError in production code

• Python 3.11+
• Daytona account and API key (sign up)
• GitHub personal access token with repo scope
• Sentry Internal Integration webhook secret
• Anthropic API key

1. Clone the repository

   git clone <your-repo-url>
   cd reflex

2. Install dependencies

   make install
   # or
   pip install -r requirements.txt

3. Configure environment variables

   cp .env.example .env
   # Edit .env with your credentials

4. Create Daytona snapshot

   make snapshot

   This creates a pre-built snapshot with pytest and build dependencies.

Create a .env file with the following:

# Daytona
DAYTONA_API_KEY=your_daytona_api_key
DAYTONA_API_URL=https://app.daytona.io/api
DAYTONA_TARGET=us

# Sentry
SENTRY_WEBHOOK_SECRET=your_webhook_secret
SENTRY_AUTH_TOKEN=your_sentry_auth_token  # Optional: for release mapping

# GitHub
GITHUB_OWNER=your_github_org
GITHUB_REPO=your_repo_name
GITHUB_TOKEN=your_github_pat

# Anthropic
ANTHROPIC_API_KEY=your_anthropic_key
ANTHROPIC_MODEL=claude-sonnet-4-5

Edit services.yaml to map service names to repositories:

my-api-service:
  repo: myorg/my-api
  path: ""  # Empty for root, or "services/api" for monorepo
  test_command: "pytest -q tests/"

my-frontend:
  repo: myorg/my-frontend
  path: ""
  test_command: "npm test"

make server
# or
python -m control.server

The server runs on http://0.0.0.0:8000 with endpoints:

• GET /health - Health check
• POST /webhooks/sentry - Sentry webhook receiver

1. Go to Settings → Developer Settings → Internal Integrations
2. Create a new Internal Integration
3. Enable Issue Alerts webhook
4. Set webhook URL to https://your-domain.com/webhooks/sentry
5. Copy the webhook secret to your .env file

To enable deterministic commit routing, configure Sentry releases in your CI:

# In your CI pipeline
export SENTRY_AUTH_TOKEN=your_token
export SENTRY_ORG=your_org
export SENTRY_PROJECT=your_project

# Create release and associate commits
sentry-cli releases new $VERSION
sentry-cli releases set-commits --auto $VERSION
sentry-cli releases finalize $VERSION

All incoming webhooks are verified using HMAC-SHA256 signatures from Sentry.

• Ephemeral sandboxes: Automatically deleted after use
• Auto-stop: Sandboxes stop after 20 minutes of inactivity
• Resource limits: CPU, memory, and disk quotas
• Network controls: Optional network blocking and allowlists

• Path restrictions: Only modifies src/, tests/, app/, lib/
• Line limits: Patches capped at 150 lines
• No traversal: Blocks parent directory access (..)
• Forbidden paths: Blocks system directories (/etc, /root, etc.)

1. Webhook Received: Sentry sends issue alert with error details
2. Routing: Resolves repository and commit from:
   • Event tags (service, repo, monorepo_path)
   • Release mapping (via Sentry API)
   • Fallback to services.yaml
3. Sandbox Creation: Spins up Daytona sandbox from snapshot
4. Reproduction: Clones repo, checks out commit, runs tests
5. Patch Generation: Claude analyzes error and generates minimal diff
6. Validation: Checks patch against safety guardrails
7. Application: Applies patch with git apply
8. Testing: Re-runs tests to verify fix
9. PR Creation: Creates branch, commits changes, opens PR
10. Cleanup: Stops and deletes sandbox

The system validates fixes by:

1. Running tests before patch (should fail)
2. Applying the patch
3. Running tests after patch (must pass)
4. Only creating PR if tests pass

Logs include:

• Webhook signature verification
• Routing decisions
• Sandbox lifecycle events
• Patch generation and validation
• Test results
• PR creation status

reflex/
├── control/
│   ├── server.py          # FastAPI webhook server (HMAC verification)
│   ├── worker.py          # Main orchestration + retry logic
│   ├── daytona_client.py  # Daytona SDK wrapper with custom diff parser
│   ├── patcher.py         # Gemini API integration (patch generation)
│   ├── github_api.py      # GitHub REST API client (PR creation)
│   └── routing.py         # Repository routing (unused in hackathon)
├── sandbox/
│   ├── Dockerfile.ci      # Snapshot base image (Python 3.12 + pytest)
│   └── requirements.txt   # Test dependencies for snapshot
├── scripts/
│   ├── make_snapshot.sh   # One-time Daytona snapshot creation
│   └── test_integration.sh # End-to-end integration tests
├── examples/
│   ├── sample_buggy_app.py  # Demo app with intentional bugs
│   ├── test_sample_app.py   # Pytest suite for demo app
│   └── run_buggy_app.sh     # Trigger Sentry alert manually
├── services.yaml          # Service routing config (optional)
├── requirements.txt       # Python dependencies
├── Makefile              # Common commands (server, snapshot, test)
└── README.md

# Unit tests
pytest tests/test_patcher.py -v

# Integration test (requires Daytona + Sentry)
bash scripts/test_integration.sh

# Manual webhook test
curl -X POST http://localhost:8000/webhooks/sentry \
  -H "Content-Type: application/json" \
  -H "Sentry-Hook-Signature: <signature>" \
  -d @examples/test_webhook.py

# Enable verbose logging
export LOG_LEVEL=DEBUG
python -m control.server

# Create new snapshot
make snapshot

# List Daytona snapshots
daytona snapshot list

# Delete old snapshot
daytona snapshot delete reflex-ci

• Check SENTRY_WEBHOOK_SECRET matches your Sentry Internal Integration
• Ensure request body is raw (not parsed JSON)

• Daytona cold start can take 20-30 seconds
• Increase timeout in worker.py if needed

• Check Gemini response in logs - may have hit retry limit
• Verify test command in services.yaml is correct
• Ensure snapshot has all dependencies (requirements.txt)

• Use file-based redirection: cmd > /tmp/output.txt 2>&1
• Read with sandbox.fs.download_file(), not process.exec()

• This is expected - system now uses direct file modification
• No action needed (handled automatically)

• Multi-language support - JavaScript/TypeScript, Java, Go
• Persistent database - Track fix history and success rates
• Web dashboard - Monitor active fixes and view logs
• Slack notifications - Alert team when PRs are created
• Cost optimization - Batch multiple errors per sandbox
• Smart routing - Auto-detect repo from stack traces
• Approval workflow - Human review before PR creation
• Metrics tracking - Success rates, fix latency, cost per fix

• Create Sandbox API
• Git Operations
• File System API
• Process Execution

• Gemini API Documentation
• Safety Settings
• Model Comparison

• Issue Alert Webhooks
• Internal Integrations
• Webhook Signatures

• Pull Requests API
• Personal Access Tokens

This is a hackathon project, but contributions are welcome!

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

MIT License - see LICENSE for details

• Daytona - For providing the sandbox infrastructure
• Google AI Studio - For Gemini API access
• Sentry - For robust error tracking and webhooks
• FastAPI - For the blazing-fast webhook server

Reflex was built to solve a real problem: production bugs that sit in backlogs for weeks. By automating the entire fix workflow - from detection to PR creation - we can reduce incident response time from hours to seconds.

Built for: Daytona Hackathon 2025
By: Manu Vikash
Tech Stack: Python • FastAPI • Daytona SDK • Google Gemini • GitHub API • Sentry

⭐ Star this repo if you find it useful!
🐛 Report bugs via GitHub Issues
💬 Questions? Open a discussion

Live Demo: github.com/manuvikash/reflex-test/pull/1