[Tests] Add inference pytest marker to filter inference tests#4338
Conversation
Replace the name-based CI filtering (pytest -k 'test_inference' / -k 'not test_inference') with an explicit `pytest.mark.inference` marker, mirroring the xet/no_xet markers introduced in #4336. The six `test_inference_*.py` modules are tagged with `pytestmark = pytest.mark.inference`. CI now selects them with `-m inference` (the "Inference only" job) and excludes them elsewhere with `-m 'not inference'`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
test_inference_endpoints.py is mocked InferenceEndpoint logic, not an inference test => unmark it so it runs in the regular jobs. Also drop the now-redundant explanatory comments above each `pytestmark`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0f9cfee. Configure here.
| "Inference only") | ||
| # Run inference tests concurrently | ||
| PYTEST="$PYTEST ../tests -k 'test_inference' -n 4" | ||
| PYTEST="$PYTEST ../tests -m 'inference' -n 4" |
There was a problem hiding this comment.
Endpoints module missing inference marker
Medium Severity
The Inference-only CI job now selects tests with -m inference, but test_inference_endpoints.py never got pytestmark = pytest.mark.inference unlike the other five test_inference_*.py modules. Those endpoint tests used to match -k test_inference via the filename and no longer run in the dedicated Inference job.
Reviewed by Cursor Bugbot for commit 0f9cfee. Configure here.
Summary
Same treatment as #4336 (xet/no_xet markers), applied to inference tests.
pytest.mark.inferencemarker (registered inpyproject.toml, enforced by--strict-markers).pytestmark = pytest.mark.inference.-m inference(was-k 'test_inference')-m 'not xet and not inference'(was-m 'not xet' -k 'not test_inference')-m 'not no_xet and not inference'(was-m 'not no_xet' -k 'not test_inference')-m 'not no_xet and not inference'(was-k 'not test_inference')CONTRIBUTING.md.Intentional scope changes vs the old
-k 'test_inference'The old name-based selection matched a few tests that aren't really inference tests. These are now unmarked and run in the regular jobs instead of "Inference only":
test_inference_endpoints.py— mockedInferenceEndpointlogic, not an inference (client/providers) test, so it's deliberately left unmarked.test_hf_api.py::...::test_inference_provider_mapping_model_info/..._list_models— Hub API tests, only matched by name coincidence.test_cli.py::...::test_inference_endpoints_alias— mocked CLI unit test, only matched by name coincidence.The
inferencemarker therefore tags the 5 real inference modules (client, async client, providers, text generation, types):🤖 Generated with Claude Code
Note
Low Risk
Test orchestration and documentation only; no library or production behavior changes.
Overview
Introduces a registered
inferencepytest marker (alongside the existing Xet markers) so the Inference suite is selected explicitly instead of by filename.Five inference modules (
test_inference_client, async client, providers, text generation, types) are tagged withpytestmark = pytest.mark.inference. CI switches from-k 'test_inference'/-k 'not test_inference'to-m inferencefor the dedicated job andnot inferenceon the Xet matrix jobs (Ubuntu and Windowswith hf_xet).CONTRIBUTING.mddocuments how to run inference-only vs non-inference locally.Compared to the old name filter,
test_inference_endpoints.pyand a few Hub API / CLI tests that only matchedtest_inferencein the name stay unmarked and run in the regular jobs.Reviewed by Cursor Bugbot for commit 90c3a4c. Bugbot is set up for automated code reviews on this repo. Configure here.