feat: add configurable custom web search backend#10414
Closed
Kolektori wants to merge 1 commit into
Closed
Conversation
Add a config-driven custom backend for web_search. The backend uses a GET URL template, expects JSON, and maps results into Hermes' existing search response format. It validates the custom config up front so availability checks and runtime failures stay predictable. The tests now cover headers, template params, malformed results, limit truncation, and the main config and error paths.
5 tasks
Collaborator
|
Merged via PR #11562 which consolidates SearXNG integration from multiple community PRs. Your self-hosted-first philosophy and documentation approach informed the final implementation. Thank you for the contribution! |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds a
custombackend forweb_searchso Hermes can query any JSON search endpoint defined in~/.hermes/config.yamlThe scope is intentionally narrow:
web_searchonlyGETonlyThe
web_searchtool schema stays unchanged. This makes local or self-hosted search APIs usable through the existing tool without adding another provider-specific backend. Examples include 4get and SearXNG, but the implementation stays generic and does not hardcode either providerRelated Issue
Fixes #10284
Type of Change
Changes Made
tools/web_tools.pyto recognizeweb.backend: customweb.custom.url_template,results_path,title_field,url_field,description_field, optionalheaders, and optionaltimeout%ssubstitutionweb_searchresult formatcheck_web_api_key()can treat a valid custom config as available without an API keytests/tools/test_web_tools_config.pyto cover request formatting, headers, truncation, malformed item skipping, config validation, and runtime error pathsHow to Test
~/.hermes/config.yaml, for example:Start Hermes with the web tool enabled and run a query that triggers
web_searchConfirm the custom endpoint receives a
GETrequest with the encoded query and Hermes returns normalized results withtitle,url,description, andpositionChecklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/A. N/A for this changecli-config.yaml.exampleif I added/changed config keys — or N/A. N/A because this is a read-path config addition with no default config changeCONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/A. N/Ahttpxcalls and config parsingweb_searchschema stays unchanged by designScreenshots / Logs
Focused validation:
Note: I did not run
pytest tests/ -qfor this PR draft, so that checklist item is intentionally left unchecked