Skip to content

feat: Add inter-profile orchestration with conversation history#3754

Closed
eloklam wants to merge 5 commits into
NousResearch:mainfrom
eloklam:feature/profile-orchestration
Closed

feat: Add inter-profile orchestration with conversation history#3754
eloklam wants to merge 5 commits into
NousResearch:mainfrom
eloklam:feature/profile-orchestration

Conversation

@eloklam

@eloklam eloklam commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Add inter-profile orchestration tools enabling one Hermes profile to delegate tasks to another via HTTP, with full conversation history passing between calls.

New Tools (4 total)

Tool Purpose
profile_delegate Delegate tasks to persistent profile workers via HTTP
profile_task_check Check status of async delegated tasks
profile_task_cancel Cancel running async tasks
profile_task_search Cross-profile natural language search with ID auto-detection

Install

Enable delegation toolset in profile config:

toolsets:
- hermes-cli
- delegation

Enable api_server on worker profile:

platforms:
  api_server:
    enabled: true
    port: 8642
    host: 127.0.0.1

Usage

# In orchestrator profile (p1):
p1 chat

# Delegate a task to another profile
> Ask p2 to calculate 5 times 5
# Returns: {"status": "completed", "result": "25", "task_id": "abc123"}

# Delegate without waiting for result
> Have p2 calculate 6 times 6 in the background
# Returns: {"status": "dispatched", "task_id": "xyz789"}

# Check on a background task later
> What's the status of task xyz789?
# Returns: {"status": "completed", "result": "36"}

# Search across all your profiles for past tasks
> Find my tasks about math calculations
# Returns: [{"task_id": "abc123", "source_profile": "p1", "goal": "calculate 5*5", ...}]

# Search by task ID prefix (auto-detected)
> Look up task abc123
# Returns: [{"task_id": "abc123def", "source_profile": "p1", ...}]

# Multi-turn conversation with memory
> Ask p2 to remember Word A is ALPHA
> Now ask p2 what Word B is BETA and what was Word A?
# Remembers "ALPHA" from conversation history

# Control whether worker can spawn one-off subagents
> Have p2 refactor the auth module without spawning any one-off subagents
# Worker must do all work itself - no delegate_task spawning

Key Features

Delegated agent awareness - Delegated agents now know they're delegated agents and not talking to human users, with visual <DELEGATED from profile X | delegate_task ALLOWED> header.

Cross-Profile Search - profile_task_search searches across ALL profiles' task histories, not just the current one. Auto-detects task ID prefixes (e.g., "abc123") vs natural language queries.

Configurable Depth - Prevent infinite delegation chains with max_depth parameter (default 2).

Delegated agent control - Use allow_subagents=False to force workers to do all work themselves without spawning one-off subagents.

Awareness format

<DELEGATED from profile p1 | delegate_task ALLOWED>

YOUR TASK:
Analyze codebase for security issues

Summary: what you did, found, created, issues.
End with <TASK abc123 COMPLETED>.

Agent: "p1 (profile, not human) delegated this. I am allowed to use delegate_task."

Control Mechanisms

  • allow_subagents=True/False — choose if worker can spawn one-off subagents
  • max_depth=N — configurable delegation depth (default 2)
  • <TASK {id} COMPLETED> footer for completion correlation

Depth Tracking

  • X-Hermes-Delegate-Depth header tracks recursion level
  • X-Hermes-Max-Depth header sets limit (configurable)
  • API server blocks when depth >= max_depth

Cross-Profile Natural Language Search

  • profile_task_search("security issues auth.py") — search across all profiles
  • Each result includes source_profile field identifying which profile handled it
  • Auto-detects task IDs: Short alphanumeric queries (e.g., "abc123", "42eb") search by task_id prefix
  • FTS5-powered: Natural language search on goal + result fields with phrases, boolean, and prefix matching
  • Hybrid approach: ID-like queries use LIKE task_id% + LIKE %query% on goal/result; natural language uses full-text search

The agent uses ThreadPoolExecutor for reliability instead of aiohttp (which caused asyncio race conditions). Async tasks persist to SQLite so they survive process restarts. If a task isn't found in memory, it falls back to SQLite lookup.

Files

File Description
tools/profile_orchestrator.py Core orchestration module with HTTP dispatch, SQLite persistence
tools/delegate_tool.py profile_delegate, profile_task_check, profile_task_cancel, profile_task_search tools
gateway/platforms/api_server.py UPDATED: Delegated agent delegation awareness with <DELEGATED> header injection and depth tracking
hermes_state.py orchestration_tasks table + FTS5 index + ID detection search
toolsets.py Extended delegation toolset with new profile orchestration tools

Tests

Handler loads correctly, passes 72 consecutive calls with 100% success rate. Manual testing on Linux (Ubuntu 22.04). Uses stdlib only, no Unix-specific code.

  • 15 parallel async dispatches: ✅
  • 25 rapid sync calls: ✅
  • 15 status checks: ✅
  • 6 mixed profile tests: ✅
  • Edge cases (unicode, special chars, multiline, fibonacci n=42): ✅
  • Delegated agent awareness detection: ✅
  • <TASK {id} COMPLETED> detection: ✅
  • Depth limit enforcement at MAX_DEPTH=2: ✅
  • allow_subagents=False shows NOT ALLOWED: ✅
  • Cross-profile natural language search: ✅
  • Task ID prefix auto-detection: ✅
  • Hybrid search (ID + FTS5): ✅

Related

Notes

No configuration needed beyond enabling the toolset and api_server — just create profiles and delegate. The orchestrator automatically discovers worker profiles via list_profiles(). If a worker isn't running, it starts automatically via subprocess with HERMES_HOME override. No worker running = delegation fails gracefully with error message.

Architecture

@eloklam

eloklam commented Mar 29, 2026

Copy link
Copy Markdown
Contributor Author

I initially merged PR #3750 (per-call model/provider override on delegate_task by HenkDz) into this branch for my local testing, but have since reverted it from the remote branch.

Why: PR #3750 is a separate feature by another contributor that I have no affiliation with. It doesn't belong in my orchestration PR. My agent merged it locally to test both features together on my PC, but accidentally pushed it to GitHub. This revert restores my branch to contain only my own inter-profile orchestration work

Thank you!

@HenkDz

HenkDz commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

Got no idea what happened here.

@eloklam eloklam force-pushed the feature/profile-orchestration branch 3 times, most recently from 0671a40 to 0176061 Compare March 30, 2026 12:14
@eloklam

eloklam commented Mar 30, 2026

Copy link
Copy Markdown
Contributor Author

🆕 What's New Since Initial Push

Cross-Profile Search:

  • profile_task_search now searches across ALL profiles' task histories
  • Each result includes source_profile field to identify which profile handled the task
  • Auto-detects task ID prefixes (short alphanumeric like "abc123") for quick lookup
  • Hybrid search: ID queries use LIKE patterns; natural language uses FTS5
  • No more "can't find my task" - search from any profile finds all your tasks

Naming Convention Fixed:
All tools now follow Hermes [noun]_[verb] pattern (like session_search, memory_save):

  • profile_delegate (was delegate_to_profile)
  • profile_task_check (was check_profile_task)
  • profile_task_cancel (was cancel_profile_task)
  • profile_task_search (was search_profile_tasks)

Delegated agent awareness:

  • Agents now know they're delegated agents: <DELEGATED from profile X | delegate_task ALLOWED> header
  • No more confusion about talking to humans vs parent agents

Control Mechanisms:

  • allow_subagents=True/False — choose if worker can spawn one-off subagents
  • max_depth=N — configurable delegation depth (default 2)
  • <TASK {task_id} COMPLETED> footer for async completion detection with ID correlation

Depth Tracking:

  • X-Hermes-Delegate-Depth header tracks recursion level
  • X-Hermes-Max-Depth header sets limit (configurable, default 2)
  • API server enforces limit: blocks when depth >= max_depth
  • Transparent to agent — just sees ALLOWED/NOT ALLOWED

Natural Language Usage:
All tools support natural language prompts:

  • "Ask p2 to calculate 5 times 5" → profile_delegate
  • "What's the status of task abc123?" → profile_task_check
  • "Find my tasks about security issues" → profile_task_search
  • "Look up task abc123" → ID prefix search (auto-detected)

@eloklam eloklam force-pushed the feature/profile-orchestration branch from 0176061 to f7b8cfb Compare March 30, 2026 14:02
eloklam added 5 commits April 8, 2026 18:19
- Profile discovery and health checking

- Port resolution and gateway lifecycle

- sync/async HTTP delegation via OpenAI-compatible API

- SQLite persistence for async tasks

- Conversation history passing between profiles
…tion

- HTTP-based delegation to persistent profile workers

- Async mode with task tracking via check_profile_task/cancel_profile_task

- Conversation history extraction and passing

- allow_subagents parameter for user control
- Detect X-Hermes-Delegation headers

- Inject delegation context into user messages

- Format: <DELEGATED from profile X | delegate_task ALLOWED>

- Depth tracking for recursion prevention (MAX_DEPTH=2)

- <TASK-COMPLETED> footer requirement

- ~77% token reduction vs original format
- SQLite table for async delegation tasks

- Schema version bump to 7

- Supports task status tracking across process restarts
- delegate_task, delegate_to_profile tools

- check_profile_task, cancel_profile_task for async management

- Included in hermes-gateway toolset
@eloklam eloklam force-pushed the feature/profile-orchestration branch from f7b8cfb to 9d0188d Compare April 8, 2026 16:34
@eloklam eloklam closed this Apr 17, 2026
@eloklam eloklam deleted the feature/profile-orchestration branch April 17, 2026 21:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants