Greptile Summary

This PR implements Phase 1 of the async subagents design (#554), adding three key capabilities to enable fire-and-forget-then-get-alerted workflow:

1. Subprocess Execution Mode

• New use_subprocess=True parameter runs subagents as isolated processes instead of threads
• Captures stdout/stderr separately via subprocess.Popen with piped output
• Background monitor thread (_monitor_subprocess) waits for completion and caches results

2. Hook-Based Completion Notifications

• _subagent_completion_hook registered via ToolSpec.hooks (not as separate file)
• Delivers system messages during LOOP_CONTINUE when subagents finish
• Thread-safe _completion_queue enables non-blocking notification delivery
• Removes need for active polling - orchestrator continues work and gets alerted

3. Batch Execution Helper

• subagent_batch() spawns multiple subagents in parallel, returns BatchJob
• BatchJob provides wait_all(), is_complete(), get_completed() for explicit synchronization
• Complements hook notifications for gather-style patterns

Infrastructure Changes

• ToolSpec gains hooks field for tool-level hook registration
• Subagent dataclass adds process, execution_mode fields
• Results cached in module-level _subagent_results dict (frozen dataclass workaround)
• Comprehensive test coverage: 15 existing tests pass + 5 new Phase 1 tests

Known Limitation (tracked in #971)

• Subprocess mode doesn't pass context_mode/context_include/output_schema to CLI
• Uses default full context for subprocess subagents
• Thread mode correctly respects all context parameters

The implementation is well-documented with clear phase roadmap and aligns with the async design document.

Confidence Score: 4/5

• Safe to merge with minor issues - well-tested Phase 1 implementation with known limitations tracked
• Score reflects solid implementation quality (comprehensive tests, clear architecture, good documentation) but deducted 1 point for: (1) subprocess cleanup not handled in test fixture leading to potential orphaned processes, (2) thread exception handling could notify with incorrect status, (3) memory concern with .communicate() for large outputs. These are minor issues that don't affect core functionality.
• Pay attention to tests/conftest.py:139-141 (subprocess cleanup) and gptme/tools/subagent.py:636-646 (exception handling in thread notification)

Important Files Changed

Sequence Diagram

sequenceDiagram
    participant Parent as Parent Agent
    participant SubagentAPI as subagent()
    participant Monitor as Monitor Thread
    participant Hook as LOOP_CONTINUE Hook
    participant Child as Child Process/Thread

    Note over Parent,Child: Phase 1: Fire-and-forget-then-get-alerted

    Parent->>SubagentAPI: subagent(agent_id, prompt, use_subprocess=True)
    SubagentAPI->>Child: spawn subprocess/thread
    SubagentAPI->>Monitor: start monitor thread
    SubagentAPI-->>Parent: returns immediately (None)
    
    Note over Parent: Parent continues working
    
    Child->>Child: execute task
    Child->>Child: call complete tool
    
    Monitor->>Child: wait for completion
    Child-->>Monitor: process exits / thread completes
    
    Monitor->>Monitor: extract result & status
    Monitor->>Monitor: cache result in _subagent_results
    Monitor->>Monitor: notify_completion(agent_id, status, summary)
    Monitor->>Hook: queue notification in _completion_queue
    
    Note over Parent,Hook: Next chat loop iteration
    
    Parent->>Hook: LOOP_CONTINUE triggered
    Hook->>Hook: _subagent_completion_hook()
    Hook->>Hook: drain _completion_queue
    Hook-->>Parent: yield system messages (✅/❌ completion)
    
    Parent->>Parent: react to completion
    
    opt Explicit synchronization
        Parent->>SubagentAPI: subagent_wait(agent_id)
        SubagentAPI->>Child: wait for thread/process
        SubagentAPI->>Monitor: retrieve cached result
        SubagentAPI-->>Parent: return status dict
    end
    
    opt Batch execution
        Parent->>SubagentAPI: subagent_batch(tasks)
        SubagentAPI->>Child: spawn multiple subagents
        SubagentAPI-->>Parent: return BatchJob
        Parent->>Parent: continue work
        Hook-->>Parent: deliver multiple completions
        opt Wait for all
            Parent->>SubagentAPI: job.wait_all()
            SubagentAPI-->>Parent: all results
        end
    end

Codecov Report

❌ Patch coverage is 41.77215% with 92 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

Looks interesting but not sure it quite succeeds in what we were trying to achieve, see my comment here: #554 (comment)

Hook-Based Refactoring Complete 🔄

I've refactored the implementation based on your feedback to use gptme's hook system instead of custom callbacks.

Key Changes

Removed:

• on_complete and on_progress callback parameters
• Callback fields from Subagent dataclass
• Manual callback invocation in monitor threads

Added:

• gptme/hooks/subagent_completion.py - New LOOP_CONTINUE hook
• notify_completion() function for thread-safe notification
• Hook registered as subagent_completion in init_hooks()
• Result summarization for token-efficient notifications

How It Works Now

# Fire-and-forget pattern
subagent('impl', 'Implement feature X')
subagent('test', 'Write tests')
subagent('docs', 'Document feature')

# Orchestrator continues with other work...
# During each loop iteration, LOOP_CONTINUE hook checks completion queue
# and delivers messages like:
#   System: ✅ Subagent 'impl' completed: Feature implemented in src/feature.py
#   System: ✅ Subagent 'test' completed: 5 tests added, all passing

Benefits

1. Uses gptme hooks ✓ - Integrated with existing hook system
2. Fire-and-forget-then-get-alerted ✓ - No polling required
3. Token-efficient ✓ - Results summarized before notification (max 200 chars)

All existing tests pass + updated test to verify new API.

🔄 CI Rerun Triggered

The failing Test with openai/gpt-4o-mini job failed due to a GitHub Actions infrastructure issue:

System.IO.IOException: No space left on device

The test suite itself passed successfully (✓ Run tests), but the runner ran out of disk space during cleanup. This is a transient infrastructure issue, not a code problem.

I've triggered a rerun of the failed job which should pass on a fresh runner with available disk space.

Looks like there are a few issues preventing this PR from being merged!

• GitHub Actions are failing:
  • Docs

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #962 at branch `feat/async-subagents`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

✅ Addressed Erik's comment about documenting the subagent_completion hook in commit 1748d53.

Changes:

• Updated module docstring with "Hook System Integration" section explaining how the subagent_completion hook works
• Moved hook functionality into the tool module (_completion_queue, notify_completion(), hook function)
• Hook now registered via ToolSpec.hooks field
• Removed standalone registration from hooks/__init__.py

The hook is now automatically registered when the subagent tool is loaded, keeping all subagent functionality together.

@greptileai review

Review Comments Addressed ✅

@ErikBjare - Addressed your review comments in commit cb7fe88:

1. Removed unused hook file

✅ Deleted gptme/hooks/subagent_completion.py - hook functionality is now in gptme/tools/subagent.py via ToolSpec (as documented in my earlier commit 1748d53).

2. Added actual subprocess execution tests

✅ Added 5 new integration tests that actually execute subprocess mode (not mocked):

• test_subprocess_actual_process_creation: Verifies real subprocess.Popen object is created
• test_subprocess_command_includes_required_flags: Verifies gptme CLI flags (-n, --no-confirm, --logdir, --model)
• test_subprocess_working_directory: Verifies workspace cwd is set correctly
• test_subprocess_full_flow_with_subagent_function: Full flow test using subagent() function
• test_subprocess_monitor_thread_started: Verifies monitor thread starts for subprocess

Test Results: All 32 tests pass (was 27)

Ready for re-review!

✅ Added subprocess mode execution tests per review feedback

Commit a55949e adds two new tests:

• test_subprocess_mode_execution_basic: Actually runs a subagent in subprocess mode and waits for completion
• test_subprocess_mode_read_log: Verifies log reading works with subprocess mode

Both tests are marked @pytest.mark.slow and @pytest.mark.eval since they actually execute gptme.

Re: hook docstring - The docstring already mentions the subagent_completion hook (lines 10-17) and the hook is already registered via ToolSpec (line 992) as suggested.

@greptileai review

✅ Greptile Review Feedback Addressed

Commit 351c217 addresses all 3 Greptile review comments:

1. Memory optimization in subprocess monitoring (line 417)

• Changed from .communicate() to .wait() to avoid reading all output into memory
• Now relies on log file for result extraction (which is the canonical source for subprocess mode)
• Prevents memory issues with long-running subagents

2. Subprocess cleanup in test fixture (line 141)

• Added subprocess termination logic to cleanup_subagents_after fixture
• Graceful termination with 5s timeout, falls back to .kill() if needed
• Ensures clean test isolation for subprocess mode tests

3. Exception handling in thread mode (line 646)

• Wrapped _create_subagent_thread() in try-except block
• If execution fails, properly notifies with 'error' status instead of running notification with incomplete state
• Prevents misleading status reports from failed subagents

All 34 subagent tests pass ✅

Note: Bot review threads show as 'unresolved' due to GitHub's review system - only maintainers can manually resolve bot reviews.