Skip to content

[BUG][API]: Forwarded RPC non-2xx responses masked as success β€” affinity tests non-hermeticΒ #3365

@crivetimihai

Description

@crivetimihai

🐞 Bug Summary

Forwarded RPC execution can incorrectly return a success-shaped payload ({"result": {}}) when the internal /rpc call fails with non-JSON-RPC HTTP errors (for example 401 {"detail":"Authorization token required"}).

This currently makes auth/policy failures look like successful forwarded execution.

Related test stability issue: two session-affinity tests depend on localhost port state (127.0.0.1:4444) and are non-hermetic.


🧩 Affected Component

Select the area of the project impacted:

  • mcpgateway - API
  • mcpgateway - UI (admin panel)
  • mcpgateway.wrapper - stdio wrapper
  • Federation or Transports
  • CLI, Makefiles, or shell scripts
  • Container setup (Docker/Podman/Compose)
  • Other (explain below)

Other: session-affinity test reliability (tests/e2e/test_session_pool_e2e.py).


πŸ” Steps to Reproduce

  1. Run these tests:
    uv run pytest -vv \
      tests/e2e/test_session_pool_e2e.py::TestMultiWorkerSessionAffinityE2E::test_execute_forwarded_request_returns_error_when_no_server \
      tests/e2e/test_session_pool_e2e.py::TestMultiWorkerSessionAffinityE2E::test_affinity_logs_when_executing_forwarded_request
  2. Ensure something is listening on 127.0.0.1:4444 and /rpc returns 401 with body shaped like {"detail": "..."} (non-JSON-RPC error object).
  3. Re-run the same tests.

Observed: both tests fail with assert "error" in result because result is {"result": {}}.

Control: with nothing listening on :4444, same tests pass (connection error path returns JSON-RPC internal error).


πŸ€” Expected Behavior

  • _execute_forwarded_request() should treat non-2xx internal /rpc responses as errors.
  • If response body is JSON-RPC ({"error": ...}), propagate it.
  • If response body is non-JSON-RPC ({"detail": ...}), map to JSON-RPC error instead of returning {"result": {}}.
  • Tests should be deterministic and not depend on ambient localhost listeners.

πŸ““ Logs / Error Output

Representative failure logs:

HTTP Request: POST http://127.0.0.1:4444/rpc "HTTP/1.0 401 Unauthorized"
[AFFINITY] ... Forwarded execution completed successfully
E AssertionError: assert 'error' in {'result': {}}

Relevant code paths:

  • mcpgateway/services/mcp_session_pool.py (_execute_forwarded_request)
  • mcpgateway/main.py (handle_rpc forwarded response handling)
  • tests/e2e/test_session_pool_e2e.py (two tests above)

🧠 Environment Info

You can retrieve most of this from the /version endpoint.

Key Value
Version or commit main@ce02c5eee
Runtime Python 3.13.7, pytest 8.4.2
Platform / OS Linux
Container none

🧩 Additional Context (optional)

Current behavior in _execute_forwarded_request() inspects JSON body for an error key but does not gate on HTTP status. That allows non-2xx + non-JSON-RPC bodies to be converted to {"result": {}}.

Suggested direction:

  • Gate on response.status_code >= 400.
  • Convert non-JSON-RPC non-2xx responses to JSON-RPC error.
  • Update tests to mock internal HTTP response shapes instead of relying on localhost process state.

Metadata

Metadata

Assignees

Labels

MUSTP1: Non-negotiable, critical requirements without which the product is non-functional or unsafeapiREST API Related itembugSomething isn't workingpythonPython / backend development (FastAPI)securityImproves securitytestingTesting (unit, e2e, manual, automated, etc)

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions