Skip to content

fix(fallback): resolve base_url from custom_providers when fallback entry has none (#15743)#15756

Closed
briandevans wants to merge 1 commit into
NousResearch:mainfrom
briandevans:fix/fallback-custom-provider-base-url-15743
Closed

fix(fallback): resolve base_url from custom_providers when fallback entry has none (#15743)#15756
briandevans wants to merge 1 commit into
NousResearch:mainfrom
briandevans:fix/fallback-custom-provider-base-url-15743

Conversation

@briandevans

Copy link
Copy Markdown
Contributor

Problem

When fallback_providers references a provider defined in custom_providers but does not supply an inline base_url, _try_activate_fallback() passed explicit_base_url=None to resolve_provider_client(). The resolution then fell through to the primary provider's endpoint, silently misdirecting the fallback request.

Reported in #15743.

Root cause

_try_activate_fallback() only sourced fb_base_url_hint from the fallback entry's own base_url field and from KNOWN_PROVIDERS. Named custom_providers entries were never consulted, so any provider whose connectivity lives entirely in custom_providers (no built-in entry) produced a None hint.

Fix

Before calling resolve_provider_client(), when neither fb_base_url_hint nor fb_api_key_hint is set, attempt a lookup via _get_named_custom_provider(fb_provider) and propagate base_url, api_key, and key_env as explicit hints:

if not fb_base_url_hint and not fb_api_key_hint:
    try:
        from hermes_cli.runtime_provider import _get_named_custom_provider
        _cp = _get_named_custom_provider(fb_provider)
        if _cp:
            fb_base_url_hint = (_cp.get("base_url") or "").strip() or None
            _cp_key = (_cp.get("api_key") or "").strip()
            _cp_key_env = (_cp.get("key_env") or "").strip()
            if not fb_api_key_hint:
                if _cp_key:
                    fb_api_key_hint = _cp_key or None
                elif _cp_key_env:
                    fb_api_key_hint = os.getenv(_cp_key_env, "").strip() or None
    except Exception:
        pass

The entire block is wrapped in except Exception: pass so a config-load failure degrades gracefully — resolve_provider_client() still runs with whatever it can derive from the provider name alone, exactly as before.

Precedence is preserved: the lookup only runs when both hints are empty. An inline base_url in the fallback entry always takes priority.

Tests

Three new tests in TestCustomProviderFallbackBaseUrl (appended to tests/run_agent/test_provider_fallback.py):

Test Assertion
test_custom_provider_base_url_used_when_not_in_fallback_entry explicit_base_url is the custom provider's URL; agent.base_url is updated; agent.provider matches
test_inline_base_url_takes_precedence_over_custom_providers_lookup _get_named_custom_provider is not called when inline base_url is present
test_custom_provider_lookup_failure_does_not_crash_fallback A RuntimeError from the lookup does not abort fallback activation

All 22 tests in the file pass (22 passed).

Checklist

  • Fixes the reported symptom end-to-end (wrong endpoint → correct endpoint)
  • Inline base_url continues to take precedence (no behaviour change for existing configs)
  • Lookup failure is silently swallowed — no regression for providers that aren't in custom_providers
  • Three targeted tests cover the happy path, the no-op path, and the error path
  • No new dependencies introduced

Closes #15743

Copilot AI review requested due to automatic review settings April 25, 2026 18:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes provider fallback activation so that when a fallback entry references a provider defined in custom_providers (or providers) but omits an inline base_url, the fallback resolves and uses the custom provider’s configured endpoint instead of silently inheriting the primary provider’s base_url (issue #15743).

Changes:

  • Extend _try_activate_fallback() to consult named custom provider config for base_url/API key hints when the fallback entry provides neither.
  • Ensure inline fallback base_url continues to take precedence (no custom provider lookup in that case).
  • Add targeted regression tests covering the happy path, precedence behavior, and lookup failure resilience.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
run_agent.py Adds a guarded lookup to pull base_url/key hints from named custom provider config before constructing the fallback client.
tests/run_agent/test_provider_fallback.py Adds regression tests verifying correct endpoint selection, precedence, and graceful handling of lookup errors.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent loop, run_agent.py, prompt builder area/config Config system, migrations, profiles labels Apr 25, 2026
…lback entry (NousResearch#15743)

When a fallback_providers entry names a provider defined in
custom_providers but does not supply an inline base_url, _try_activate_fallback()
was silently sending the request to the primary provider's endpoint instead of
the custom one.

Fix: before calling resolve_provider_client, look up the provider name in
custom_providers via _get_named_custom_provider() and propagate its base_url
(and api_key / key_env) as explicit hints. The lookup is wrapped in a broad
except so a config-load failure degrades gracefully to the existing
named-provider path instead of aborting the fallback chain.

Adds three new tests in TestCustomProviderFallbackBaseUrl covering:
- base_url pulled from custom_providers when absent from fallback entry
- inline base_url on the fallback entry skips the lookup entirely
- lookup failure does not crash fallback activation

Closes NousResearch#15743

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@briandevans

Copy link
Copy Markdown
Contributor Author

Closing — issue #15743 was marked completed on 2026-05-01 and several fallback/credential_pool fixes have since landed on main (#22780, #22842, #21219, #6ddc48b05). Happy to reopen if this specific gap still reproduces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent loop, run_agent.py, prompt builder P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fallback provider defined in custom_providers ignores its own base_url, sends request to primary provider's endpoint instead

3 participants