Skip to content

feat: add workspace foundation and RAG retrieval system#5619

Closed
teknium1 wants to merge 1 commit into
mainfrom
hermes/hermes-8cc55056
Closed

feat: add workspace foundation and RAG retrieval system#5619
teknium1 wants to merge 1 commit into
mainfrom
hermes/hermes-8cc55056

Conversation

@teknium1

@teknium1 teknium1 commented Apr 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Port and modernize PR #1324 onto current main with full profile/HERMES_HOME awareness.

Brings the workspace + knowledgebase RAG system to a usable state. Users can drop documents into ~/.hermes/workspace/ (or any profile's workspace directory), and the system indexes, chunks, and retrieves relevant context at query time.

What this PR does

New files (3):

  • agent/workspace.py — Core workspace engine: path resolution, manifest generation, structural chunking (markdown heading-aware, code symbol-aware), chunk indexing into SQLite, hybrid retrieval (FTS5 sparse + dense embeddings via RRF), optional reranking (local cross-encoder, Cohere, Voyage, heuristic fallback), workspace roots management, turn-scoped context injection
  • tools/workspace_tool.py — Model-facing workspace tool (status/index/list/search/retrieve)
  • hermes_cli/workspace.py — CLI subcommands and /workspace slash command handler

Integration points (9 existing files):

  • config.py: workspace + knowledgebase sections in DEFAULT_CONFIG, dirs in ensure_hermes_home(), version bump to 13
  • toolsets.py: workspace tool in _HERMES_CORE_TOOLS
  • model_tools.py: workspace_tool in _discover_tools()
  • commands.py: /workspace CommandDef
  • cli.py: /workspace slash command dispatch
  • run_agent.py: turn-scoped workspace RAG context injection (cache-safe — appended to current-turn user message only, never touches system prompt)
  • main.py: hermes workspace subcommand tree (status/index/list/search/retrieve/roots)
  • banner.py: workspace roots visibility in welcome banner
  • pyproject.toml: workspace-rag optional dependency group

Key design decisions

  • Profile-aware: all paths use get_hermes_home() from hermes_constants. Each profile gets its own workspace/ and knowledgebase/ directories.
  • Cache-safe injection: workspace context is appended to the current-turn user message only (same pattern as Honcho), never modifying the system prompt or cached prefix.
  • Graceful degradation: works with zero optional deps (hash embeddings + Python cosine similarity). SentenceTransformers + sqlite-vec are optional accelerators.
  • Retrieval modes: off (default), gated (heuristic trigger), always. Default is off — users opt in.
  • Workspace roots: canonical workspace + additional directories configurable via hermes workspace roots add.

CLI interface

hermes workspace status          # Show workspace and index info
hermes workspace index           # Rebuild chunk index
hermes workspace list [path]     # List workspace files
hermes workspace search <query>  # Regex search workspace files
hermes workspace retrieve <query># Hybrid RAG retrieval
hermes workspace roots list      # Show active roots
hermes workspace roots add <path> [--recursive]
hermes workspace roots remove <path-or-label>
/workspace [status|index|list|search|retrieve|roots]  # Slash command

Test plan

  • 18 new workspace-specific tests (all passing)
  • 130 toolset/model_tools/commands tests — passing
  • 227 run_agent tests — passing
  • 66 cli/config tests — passing
  • E2E verification: isolated HERMES_HOME, real file I/O, full index→retrieve→context-inject pipeline

Original draft: #1324

@teknium1
feat: add workspace foundation and RAG retrieval system
81d4467 
Port and modernize PR #1324 onto current main with full profile/HERMES_HOME awareness.

New files:
- agent/workspace.py: Core workspace engine — path resolution, manifest generation,
  structural chunking (markdown heading-aware, code symbol-aware), chunk indexing
  into SQLite, hybrid retrieval (FTS5 sparse + dense embeddings via RRF), optional
  reranking (local cross-encoder, Cohere, Voyage, heuristic fallback), workspace
  roots management, turn-scoped context injection
- tools/workspace_tool.py: Model-facing workspace tool (status/index/list/search/retrieve)
- hermes_cli/workspace.py: CLI subcommands and /workspace slash command handler

Integration points:
- config.py: workspace and knowledgebase sections in DEFAULT_CONFIG, workspace/
  knowledgebase dirs in ensure_hermes_home(), config version bump to 13
- toolsets.py: workspace tool added to _HERMES_CORE_TOOLS
- model_tools.py: workspace_tool added to _discover_tools()
- commands.py: /workspace CommandDef with subcommands
- cli.py: /workspace slash command dispatch
- run_agent.py: turn-scoped workspace RAG context injection (cache-safe —
  appended to current-turn user message only, never touches system prompt)
- hermes_cli/main.py: hermes workspace subcommand tree
  (status/index/list/search/retrieve/roots)
- hermes_cli/banner.py: workspace roots visibility in welcome banner
- pyproject.toml: workspace-rag optional dependency group

Profile-aware: all paths use get_hermes_home() from hermes_constants,
never hardcoded ~/.hermes. Each profile gets its own workspace/ and
knowledgebase/ directories.

Retrieval modes: off (default), gated (heuristic trigger), always.
Embedding: local SentenceTransformers when installed, hash fallback otherwise.
Dense search: sqlite-vec acceleration when installed, Python cosine fallback.

Tests: 18 new workspace-specific tests, all passing.
Original PR: #1324 by @teknium1
@github-actions

github-actions Bot commented Apr 6, 2026

Copy link
Copy Markdown
Contributor

⚠️ Supply Chain Risk Detected

This PR contains patterns commonly associated with supply chain attacks. This does not mean the PR is malicious — but these patterns require careful human review before merging.

⚠️ WARNING: exec() or eval() usage

Dynamic code execution can hide malicious behavior, especially when combined with base64 or network fetches.

Matches (first 20):

1578:+def _should_attempt_workspace_retrieval(user_message: str) -> bool:
1600:+    if mode == "gated" and not _should_attempt_workspace_retrieval(user_message):

⚠️ WARNING: Outbound network calls (POST/PUT)

Outbound POST/PUT requests in new code could be data exfiltration. Verify the destination URLs are legitimate.

Matches (first 10):

833:+                response = requests.post(
958:+            response = requests.post(
984:+            response = requests.post(

Automated scan triggered by supply-chain-audit. If this is a false positive, a maintainer can approve after manual review.

@kshitijk4poor kshitijk4poor mentioned this pull request Apr 7, 2026
Closed
kshitijk4poor added a commit to kshitijk4poor/hermes-agent that referenced this pull request Apr 7, 2026
@kshitijk4poor
Modularize workspace retrieval so the salvaged PR stays maintainable
30cc557 
The original workspace foundation commit ports cleanly onto current main,
but it still carried a large monolithic indexing/retrieval implementation.
This follow-up extracts the pipeline into plugin contracts, built-in plugin
implementations, and a manager that resolves category-specific backends while
preserving the existing workspace CLI/tool behavior.

Constraint: Salvage PR NousResearch#5619 on top of current upstream main without regressing workspace behavior
Rejected: Keep the monolithic workspace engine and merge as-is | leaves duplicated indexing/retrieval logic and makes backend swaps harder
Rejected: Rewrite the workspace feature from scratch on upstream main | unnecessary scope and higher merge risk
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Keep workspace behavior tests plugin-facing; do not reintroduce private helper-only assertions for indexing/retrieval internals
Tested: uv run --extra dev python -m pytest -q tests/agent/test_workspace.py tests/tools/test_workspace_tool.py tests/test_workspace_cli_command.py tests/agent/test_workspace_plugins.py
Not-tested: Full repo pytest, lint, and typecheck
Related: PR NousResearch#5619
kshitijk4poor added a commit to kshitijk4poor/hermes-agent that referenced this pull request Apr 7, 2026
@kshitijk4poor
Modularize workspace retrieval so the salvaged PR stays maintainable
6ed2f06 
The original workspace foundation commit ports cleanly onto current main,
but it still carried a large monolithic indexing/retrieval implementation.
This follow-up extracts the pipeline into plugin contracts, built-in plugin
implementations, and a manager that resolves category-specific backends while
preserving the existing workspace CLI/tool behavior.

Constraint: Salvage PR NousResearch#5619 on top of current upstream main without regressing workspace behavior
Rejected: Keep the monolithic workspace engine and merge as-is | leaves duplicated indexing/retrieval logic and makes backend swaps harder
Rejected: Rewrite the workspace feature from scratch on upstream main | unnecessary scope and higher merge risk
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Keep workspace behavior tests plugin-facing; do not reintroduce private helper-only assertions for indexing/retrieval internals
Tested: uv run --extra dev python -m pytest -q tests/agent/test_workspace.py tests/tools/test_workspace_tool.py tests/test_workspace_cli_command.py tests/agent/test_workspace_plugins.py
Not-tested: Full repo pytest, lint, and typecheck
Related: PR NousResearch#5619
@alt-glitch alt-glitch mentioned this pull request Apr 17, 2026
Closed
19 tasks
@teknium1

teknium1 commented Apr 19, 2026

Copy link
Copy Markdown
Contributor Author

Closing during PR triage — not pursuing this approach.

@teknium1 teknium1 closed this Apr 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@teknium1