refactor: move all source into hermes_agent/ package (PR 1/3)#14498
refactor: move all source into hermes_agent/ package (PR 1/3)#14498alt-glitch wants to merge 22 commits into
Conversation
…level Sweep ~74 redundant local imports across 21 files where the same module was already imported at the top level. Also includes type fixes and lint cleanups on the same branch.
Full AST-based scan of all .py files to find every case where a module
or name is imported locally inside a function body but is already
available at module level. This is the second pass — the first commit
handled the known cases from the lint report; this one catches
everything else.
Files changed (19):
cli.py — 16 removals: time as _time/_t/_tmod (×10),
re / re as _re (×2), os as _os, sys,
partial os from combo import,
from model_tools import get_tool_definitions
gateway/run.py — 8 removals: MessageEvent as _ME /
MessageType as _MT (×3), os as _os2,
MessageEvent+MessageType (×2), Platform,
BasePlatformAdapter as _BaseAdapter
run_agent.py — 6 removals: get_hermes_home as _ghh,
partial (contextlib, os as _os),
cleanup_vm, cleanup_browser,
set_interrupt as _sif (×2),
partial get_toolset_for_tool
hermes_cli/main.py — 4 removals: get_hermes_home, time as _time,
logging as _log, shutil
hermes_cli/config.py — 1 removal: get_hermes_home as _ghome
hermes_cli/runtime_provider.py
— 1 removal: load_config as _load_bedrock_config
hermes_cli/setup.py — 2 removals: importlib.util (×2)
hermes_cli/nous_subscription.py
— 1 removal: from hermes_cli.config import load_config
hermes_cli/tools_config.py
— 1 removal: from hermes_cli.config import load_config, save_config
cron/scheduler.py — 3 removals: concurrent.futures, json as _json,
from hermes_cli.config import load_config
batch_runner.py — 1 removal: list_distributions as get_all_dists
(kept print_distribution_info, not at top level)
tools/send_message_tool.py
— 2 removals: import os (×2)
tools/skills_tool.py — 1 removal: logging as _logging
tools/browser_camofox.py
— 1 removal: from hermes_cli.config import load_config
tools/image_generation_tool.py
— 1 removal: import fal_client
environments/tool_context.py
— 1 removal: concurrent.futures
gateway/platforms/bluebubbles.py
— 1 removal: httpx as _httpx
gateway/platforms/whatsapp.py
— 1 removal: import asyncio
tui_gateway/server.py — 2 removals: from datetime import datetime,
import time
All alias references (_time, _t, _tmod, _re, _os, _os2, _json, _ghh,
_ghome, _sif, _ME, _MT, _BaseAdapter, _load_bedrock_config, _httpx,
_logging, _log, get_all_dists) updated to use the top-level names.
Widen return type annotations to match actual control flow, add unreachable assertions after retry loops ty cannot prove terminate, split ambiguous union returns (auth.py credential pool), and remove the AIOHTTP_AVAILABLE conditional-import guard from api_server.py.
Move batch_runner, trajectory_compressor, mini_swe_runner, and rl_cli from the project root into scripts/, update all imports, logger names, pyproject.toml, and downstream test references.
Replace hasattr() duck-typing with isinstance() checks for DiscordAdapter in gateway/run.py, add TypedDict for IMAGEGEN_BACKENDS in tools_config.py, properly type fal_client getattr'd callables in image_generation_tool.py, fix dict[str, object] → Callable annotation in approval.py, use isinstance(BaseModel) in web_tools.py, capture _message_handler to local in base.py, rename shadowed list_distributions parameter in batch_runner.py, and remove dead queue_message branch.
…guards Previously mutagen, aiohttp-socks, tiktoken, Pillow, psutil, datasets, neutts, and soundfile were used behind try/except ImportError with silent fallbacks, masking broken functionality at runtime. Declare each in its natural extra (messaging, cli, mcp, rl, new tts-local) so they get installed, and remove the guards so missing deps crash loudly.
Add TypedDicts for DEFAULT_CONFIG, CLI state dicts (_ModelPickerState, _ApprovalState, _ClarifyState), and OPTIONAL_ENV_VARS so ty can resolve nested dict subscripts. Guard Optional returns before subscripting (toolsets, cron/scheduler, delegate_tool), coerce str|None to str before slicing (gateway/run, run_agent), split ternary for isinstance narrowing (wecom), and suppress discord interaction.data access with ty: ignore.
15 P1 ship-stopper runtime bugs from the ty triage plus the cross-bucket cleanup in run_agent.py. Net: -138 ty diagnostics (1953 -> 1815). Major wins on not-subscriptable (-34), unresolved-attribute (-29), invalid-argument-type (-26), invalid-type-form (-20), unsupported-operator (-18), invalid-key (-9). Missing refs (structural): - tools/rl_training_tool.py: RunState dataclass gains api_log_file, trainer_log_file, env_log_file fields; stop-run was closing undeclared handles. - agent/credential_pool.py: remove_entry(entry_id) added, symmetric with add_entry; used by hermes_cli/web_server.py OAuth dashboard cleanup. - hermes_cli/config.py: _CamofoxConfig TypedDict defined (was referenced by _BrowserConfig but never declared). - hermes_cli/gateway.py: _setup_wecom_callback() added, mirroring _setup_wecom(). - tui_gateway/server.py: skills_hub imports corrected from hermes_cli.skills_hub -> tools.skills_hub. Typo / deprecation: - tools/transcription_tools.py: os.sys.modules -> sys.modules. - gateway/platforms/bluebubbles.py: datetime.utcnow() -> datetime.now(timezone.utc). None-guards: - gateway/platforms/telegram.py:~2798 - msg.sticker None guard. - gateway/platforms/discord.py:3602/3637 - interaction.data None + SelectMenu narrowing; :3009 - thread_id None before `in`; :1893 - guild.member_count None. - gateway/platforms/matrix.py:2174/2185 - walrus-narrow re.search().group(). - agent/display.py:732 - start_time None before elapsed subtraction. - gateway/run.py:10334 - assert _agent_timeout is not None before `// 60`. Platform override signature match: - gateway/platforms/email.py: send_image accepts metadata kwarg; send_document accepts **kwargs (matches base class). run_agent.py annotation pass: - callable/any -> Callable/Any in annotation position (15 sites in run_agent.py + 5 in cli.py, toolset_distributions.py, tools/delegate_tool.py, hermes_cli/dingtalk_auth.py, tui_gateway/server.py). - conversation_history param widened to list[dict[str, Any]] | None. - OMIT_TEMPERATURE sentinel guarded from leaking into call_llm(temperature): kwargs-dict pattern at run_agent.py:7337 + scripts/trajectory_compressor.py:618/688. - build_anthropic_client(timeout) widened to Optional[float]. Tests: - tests/agent/test_credential_pool.py: remove_entry (id match, unknown-id, priority renumbering). - tests/hermes_cli/test_config_shapes.py: _CamofoxConfig shape + nesting. - tests/tools/test_rl_training_tool.py: RunState log_file fields.
# Conflicts: # gateway/platforms/base.py # gateway/platforms/qqbot/adapter.py # gateway/platforms/slack.py # hermes_cli/main.py # scripts/batch_runner.py # tools/skills_tool.py # uv.lock
Follow-up to 15ac253 per /simplify review: - gateway/platforms/discord.py:3638 - move self.resolved = True *after* the `if interaction.data is None: return` guard. Previously the view was marked resolved before the None-guard, so a None data payload silently rejected the user's next click. - agent/display.py:732 - replace `if self.start_time is None: continue` with `assert self.start_time is not None`. start() sets start_time before the animate thread starts, so the None branch was dead; the `continue` form would have busy-looped (skipping the 0.12s sleep). - tests/hermes_cli/test_config_shapes.py - drop __total__ dunder restatement test (it just echoes the class declaration); trim commit narration from module docstring. - tests/agent/test_credential_pool.py, tests/tools/test_rl_training_tool.py - drop "added in commit ..." banners (narrates the change per CLAUDE.md).
# Conflicts: # gateway/run.py # tools/delegate_tool.py
When optional dependencies are missing, raise ImportError with installation instructions pointing to the relevant extras group (e.g. `[messaging]`, `[cli]`, `[mcp]`, etc.) instead of letting the import fail silently.
Remove the unnecessary nudge about agent refactoring; the TODO describes the actual work that needs to be done.
…kage Pure file moves, zero content changes. Every file in agent/, tools/, hermes_cli/, gateway/, acp_adapter/, cron/, and plugins/ moves into hermes_agent/. Top-level modules (run_agent.py, cli.py, etc.) move to their new homes per the restructure manifest. Git sees 100% similarity on all moves. Part of #14182, #14183
Rewrite all import statements, patch() targets, sys.modules keys, importlib.import_module() strings, and subprocess -m references to use hermes_agent.* paths. Strip sys.path.insert hacks from production code (rely on editable install). Update COMPONENT_PREFIXES for logger filtering. Fix 3 hardcoded getLogger() calls to use __name__. Update transport and tool registry discovery paths. Update plugin module path strings. Add legacy process-name patterns for gateway PID detection. Add main() to skills_sync for console_script entry point. Fix _get_bundled_dir() path traversal after move. Part of #14182, #14183
Fix variable name breakage (run_agent, hermes_constants, etc.) where import rewriter changed 'import X' to 'import hermes_agent.Y' but test code still referenced 'X' as a variable name. Fix package-vs-module confusion (cli.auth, cli.models, cli.ui) where single files became directories. Fix hardcoded file paths in tests pointing to old locations. Fix tool registry to discover tools in subpackage directories. Fix stale import in hermes_agent/tools/__init__.py. Part of #14182, #14183
Update pyproject.toml entry points, packages.find, and package-data. Delete py-modules (all top-level modules moved into hermes_agent/). Add hermes-skills-sync console_script entry point. Update Dockerfile HERMES_WEB_DIST path. Update docker/entrypoint.sh, scripts/install.sh, setup-hermes.sh to use hermes-skills-sync console_script. Update web/vite.config.ts output directory. Update MANIFEST.in to graft hermes_agent. Update AGENTS.md project structure to reflect new layout. Part of #14182, #14183
- plugins_cmd.py: import rewriter changed `import hermes_cli` to `import hermes_agent.cli` but left variable usage as `hermes_cli.__file__`, causing a NameError at runtime - scripts/hermes-gateway: stale `from gateway.run import` (no .py extension so it was missed by **/*.py globs) - scripts/install.ps1: stale `tools\skills_sync.py` path, use hermes-skills-sync console_script instead
| return {} | ||
| try: | ||
| from hermes_cli.config import load_config | ||
| from hermes_agent.cli.config import load_config |
Tells git blame (and GitHub) to skip the git-mv and import-rewrite commits so blame shows the original author.
🚨 CRITICAL Supply Chain Risk DetectedThis PR contains a pattern that has been used in real supply chain attacks. A maintainer must review the flagged code carefully before merging. 🚨 CRITICAL: Install-hook file added or modifiedThese files can execute code during package installation or interpreter startup. Files: Scanner only fires on high-signal indicators: .pth files, base64+exec/eval combos, subprocess with encoded commands, or install-hook files. Low-signal warnings were removed intentionally — if you're seeing this comment, the finding is worth inspecting. |
82d6a80 to
4150433
Compare
|
Superseded by the top-down granular approach. This monolithic PR (+13,057/-12,287) is replaced by 6 incremental PRs ordered by inbound dependency count. See updated tracker: #14182 Sub-issues: #14586 (acp) → #14587 (cron) → #14588 (gateway) → #14589 (cli+plugins) → #14590 (tools+backends) → #14591 (agent+providers+leaves) The branch |
Summary
Consolidates the entire codebase into a single
hermes_agent/Python package. This is PR 1 of 3 in the restructure tracked by #14182 (sub-issue: #14183).Before: 8 independent packages (
agent/,tools/,hermes_cli/,gateway/,acp_adapter/,cron/,plugins/) + 11 top-level modules (run_agent.py,cli.py,model_tools.py, etc.) with unprefixed imports (from agent.prompt_builder import ...).After: One
hermes_agent/package with organized subpackages. All imports arefrom hermes_agent.X import ....Diff breakdown
The diff looks large (+13,057 / -12,287) but ~95% is mechanical one-line import path changes. Here's the breakdown:
git mvonly — invisible in difftui_gateway/,environments/, Dockerfile, AGENTS.md, etc.Tests dominate because every test file imports from the packages we moved, so every
from agent.,from tools.,from hermes_cli.line was rewritten — plus allpatch()andmonkeypatch.setattr()target strings. There are more test files (649) than source files (291) because each test imports and mocks multiple modules.The actual semantic changes (infra, new scripts,
.git-blame-ignore-revs) account for ~1,100 lines.What changed
hermes_agent/viagit mv(100% similarity preserved for blame)import X,from X import,patch()strings,monkeypatch.setattr(),sys.modules[]keys,importlib.import_module()stringspyproject.tomlentry points,Dockerfile,docker/entrypoint.sh,scripts/install.sh,setup-hermes.sh,scripts/install.ps1,web/vite.config.ts,MANIFEST.in,AGENTS.md,nix/checks.nixagent/transports/+ provider adapters →hermes_agent/providers/tools/environments/→hermes_agent/backends/tools/browser_*.py→hermes_agent/tools/browser/tools/mcp_*.py→hermes_agent/tools/mcp/tools/skills_*.py→hermes_agent/tools/skills/tools/file_*.py→hermes_agent/tools/files/tools/*_tool.py(media) →hermes_agent/tools/media/hermes_cli/auth*.py→hermes_agent/cli/auth/hermes_cli/model*.py→hermes_agent/cli/models/hermes_cli/banner.py,colors.py, etc. →hermes_agent/cli/ui/hermes-skills-sync— decouples shell scripts from internal module layoutCommit strategy
8bff8bf265ca3ba9git mv— all files intohermes_agent/(100% similarity)4b163419a1e667b976aebd73ff99611e25072fe6987962f4.git-blame-ignore-revsfor restructure commitsCommits 1-2 are separated so
git log --followandgit blamework correctly. Thegit mvcommit has 100% similarity on all 268 files..git-blame-ignore-revstells git blame (and GitHub) to skip commits 2-3.Key decisions
plugins/locationhermes_agent/plugins/sys.pathhacks in prod codetests/conftest.pysys.pathImportErrorguarduv pip install -e '.[all,dev]'COMPONENT_PREFIXEShermes logs --componentfilteringskills_synchermes-skills-syncconsole_scriptsetup-hermes.sh,install.sh,entrypoint.shfrom internal layout_discover_transports()hermes_agent.providers.*_transportpathsdiscover_builtin_tools()browser/,files/,media/,skills/subpackage directoriesTest results
All errors are pre-existing (
Callabletype annotation bug). The +121 passed is from previously-erroring tests now resolving correctly. No regressions introduced.Entry points verified
Follow-up issues
Directories unchanged (stay at repo root)
skills/,optional-skills/,environments/,tui_gateway/,ui-tui/,web/,website/,docker/,nix/,packaging/,scripts/,tests/,acp_registry/,assets/Closes #14183
Part of #14182