fix(tests): prevent flaky test_validation by preloading SDK error cla…#632
Conversation
Greptile SummaryThis PR fixes a flaky-test race condition in Confidence Score: 5/5Safe to merge — the fix is correct, well-scoped, and addresses the root cause without touching production code. The autouse fixture correctly short-circuits both lazy loaders' import branches by pre-populating the *AuthError globals before any individual monkeypatch runs. The logic chain holds: autouse sets *AuthError → test patches Anthropic/OpenAI → loader sees no None → condition False → fakes returned, real SDK never re-injected. No P0/P1 findings; all remaining considerations are speculative or out of scope. No files require special attention. Important Files Changed
|
prevent flaky test_validation by preloading SDK error classes
_load_anthropic_client() and _load_openai_client() in app/cli/wizard/
validation.py lazily import their SDKs and set both the client class
(Anthropic/OpenAI) and the matching *AuthError global. They re-import
whenever *AuthError is None, which silently overwrites any monkeypatch
of the client class.
The tests only patched Anthropic/OpenAI (not *AuthError), so on the
first call in a pytest-xdist worker the real SDK was re-injected, the
real API was called using ANTHROPIC_API_KEY/OPENAI_API_KEY leaked in
from the CI environment, and the test failed with 'rejected the API
key'. Pass/fail depended on xdist worker distribution — any PR that
added tests could trigger it.
Add an autouse fixture that pre-populates the *AuthError globals with
the real error classes so the loaders' import branch never fires in
tests. Subsequent monkeypatches of Anthropic/OpenAI are now stable.