-
Notifications
You must be signed in to change notification settings - Fork 614
[BUG][TESTING]: Flaky tests: TTL expiration and tool listing error handling #2521
Copy link
Copy link
Closed
Copy link
Labels
WOULDP4: Not a priority for current scope; very likely to move to future releasesP4: Not a priority for current scope; very likely to move to future releasesbugSomething isn't workingSomething isn't workinggood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededpythonPython / backend development (FastAPI)Python / backend development (FastAPI)test-automationAutomated testingAutomated testingtestingTesting (unit, e2e, manual, automated, etc)Testing (unit, e2e, manual, automated, etc)
Milestone
Description
Description
Two tests were identified as flaky during PR #2507 review. They fail intermittently when run as part of a larger test suite but pass consistently when run in isolation.
Affected Tests
1. TTL Expiration Test
tests/unit/mcpgateway/plugins/plugins/response_cache_by_prompt/test_response_cache_by_prompt.py::TestEvictionAndIndexRebuild::test_ttl_expiration_and_index_rebuild
- Error:
AssertionError: assert 2 == 1 - Likely cause: Timing sensitivity - TTL-based tests are prone to race conditions
2. Tool Listing Error Handling
tests/unit/mcpgateway/services/test_tool_service.py::TestToolListingGracefulErrorHandling::test_list_tools_continues_on_conversion_error
- Error:
AssertionError: assert 1 == 2 - Likely cause: Test isolation issue - shared state from parallel test execution
Reproduction
- Run full test suite: fails intermittently
- Run tests in isolation: passes consistently (verified 6/6 runs)
Suggested Fixes
- TTL test: Add tolerance margin or use
freezegun/time-machineto mock time - Tool service test: Ensure proper test isolation with fresh mocks/fixtures
Labels
flaky-testtesting
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
WOULDP4: Not a priority for current scope; very likely to move to future releasesP4: Not a priority for current scope; very likely to move to future releasesbugSomething isn't workingSomething isn't workinggood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededpythonPython / backend development (FastAPI)Python / backend development (FastAPI)test-automationAutomated testingAutomated testingtestingTesting (unit, e2e, manual, automated, etc)Testing (unit, e2e, manual, automated, etc)