Skip to content

[BUG][TESTING]: Flaky tests: TTL expiration and tool listing error handling #2521

@crivetimihai

Description

@crivetimihai

Description

Two tests were identified as flaky during PR #2507 review. They fail intermittently when run as part of a larger test suite but pass consistently when run in isolation.

Affected Tests

1. TTL Expiration Test

tests/unit/mcpgateway/plugins/plugins/response_cache_by_prompt/test_response_cache_by_prompt.py::TestEvictionAndIndexRebuild::test_ttl_expiration_and_index_rebuild
  • Error: AssertionError: assert 2 == 1
  • Likely cause: Timing sensitivity - TTL-based tests are prone to race conditions

2. Tool Listing Error Handling

tests/unit/mcpgateway/services/test_tool_service.py::TestToolListingGracefulErrorHandling::test_list_tools_continues_on_conversion_error
  • Error: AssertionError: assert 1 == 2
  • Likely cause: Test isolation issue - shared state from parallel test execution

Reproduction

  • Run full test suite: fails intermittently
  • Run tests in isolation: passes consistently (verified 6/6 runs)

Suggested Fixes

  1. TTL test: Add tolerance margin or use freezegun/time-machine to mock time
  2. Tool service test: Ensure proper test isolation with fresh mocks/fixtures

Labels

  • flaky-test
  • testing

Metadata

Metadata

Assignees

Labels

WOULDP4: Not a priority for current scope; very likely to move to future releasesbugSomething isn't workinggood first issueGood for newcomershelp wantedExtra attention is neededpythonPython / backend development (FastAPI)test-automationAutomated testingtestingTesting (unit, e2e, manual, automated, etc)

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions