Skip to content

feat: add transform_llm_output plugin hook#20813

Closed
JezzaHehn wants to merge 1 commit into
NousResearch:mainfrom
JezzaHehn:feature/transform-llm-output-hook
Closed

feat: add transform_llm_output plugin hook#20813
JezzaHehn wants to merge 1 commit into
NousResearch:mainfrom
JezzaHehn:feature/transform-llm-output-hook

Conversation

@JezzaHehn

@JezzaHehn JezzaHehn commented May 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a new plugin hook transform_llm_output that allows plugins to transform the LLM's output text before it's returned to the user.

Motivation

Personalized vocabulary transformation previously required either lots of personality flavor text in the SOUL, a skill that burned inference tokens on every response, or similar systems which burn tokens unnecessarily. A native plugin hook allows zero-cost vocabulary/personality transformation using more "classical" programming methods.

This follows the pattern of existing transform hooks (transform_terminal_output, transform_tool_result):

  • Fired once per turn after the tool-calling loop completes
  • First non-empty string result wins
  • Fail-open: exceptions logged as warnings, don't break agent execution

Changes

hermes_cli/plugins.py  | 4 ++++
run_agent.py           | 21 +++++++++++++++++++++

hermes_cli/plugins.py

Added transform_llm_output to VALID_HOOKS set.

run_agent.py

Added hook invocation after final_response is set, before post_llm_call.

Hook Signature

def transform_llm_output(response_text: str, session_id: str, model: str, platform: str) -> str | None
  • Return: Non-empty string to transform, None or empty string to pass through
  • Priority: First non-empty string result wins (same as transform_tool_result)
  • Error handling: Fail-open, exceptions logged as warnings

Use Case

# ~/.hermes/plugins/spongebob-vocab/handler.py
import os

def transform_llm_output(response_text: str, session_id: str, model: str, platform: str) -> str | None:
    if os.environ.get("SPONGEBOB_MODE") != "on":
        return None  # Pass through unchanged
    
    # Transform exclamations in conversational parts
    # (Preserves code blocks, URLs, paths, commands)
    ...

Testing

Unit test verified:

SPONGEBOB_MODE=on python3 -c "
from hermes_cli.plugins import get_plugin_manager
pm = get_plugin_manager()
pm.discover_and_load()
results = pm.invoke_hook('transform_llm_output', 'Wow!! Amazing!', '', '', '')
print(results[0])  # Output: Tartar sauce!! Holy Krabby Patties!
"

Documentation

This hook will be documented in the plugin hooks reference alongside transform_terminal_output and transform_tool_result.

Enables plugins to transform LLM output text after generation,
useful for vocabulary/personality transformation without burning
inference tokens.

Follows same pattern as transform_tool_result and transform_terminal_output:
- First non-empty string result wins
- Fail-open: exceptions logged as warnings, agent continues
- Signature: (response_text, session_id, model, platform)
@JezzaHehn JezzaHehn marked this pull request as ready for review May 6, 2026 16:20
@JezzaHehn

JezzaHehn commented May 6, 2026

Copy link
Copy Markdown
Contributor Author

Please note the "Contributor Attribution Check" pipeline was due to my not having added my agent's email address to my GitHub account, which is now fixed

The other failed pipeline, I'm not sure if that has anything to do with this change

Big thanks as always to the core Nous team!!!! 💚

Screenshot of the silly Spongebob flavored version in action:
image

Other more useful cases might be name substitution or profanity cleaning.

@alt-glitch alt-glitch added type/feature New feature or request comp/plugins Plugin system and bundled plugins comp/agent Core agent loop, run_agent.py, prompt builder P3 Low — cosmetic, nice to have labels May 6, 2026
@JezzaHehn

JezzaHehn commented May 6, 2026

Copy link
Copy Markdown
Contributor Author

Additional small note, the following quicktip is adjacent to this PR and might could be updated if/when the PR goes through cleanly


✦ Tip: Plugin hooks include pre/post_tool_call, pre/post_llm_call, and transform_terminal_output for output canonicalization.

@teknium1

teknium1 commented May 7, 2026

Copy link
Copy Markdown
Contributor

Salvaged via PR #21235 with your authorship preserved via rebase-merge (commit c3be6ec on main). Added dispatch-semantics tests, the docs section in website/docs/user-guide/features/hooks.md, and an AUTHOR_MAP entry for your agentmail address. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent loop, run_agent.py, prompt builder comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants