fix(tests): make test_modal_terminal actually assert, not return booleans#13318
Open
Sanjays2402 wants to merge 1 commit into
Open
fix(tests): make test_modal_terminal actually assert, not return booleans#13318Sanjays2402 wants to merge 1 commit into
Sanjays2402 wants to merge 1 commit into
Conversation
…eans Closes NousResearch#13286. Six of the tests in tests/integration/test_modal_terminal.py used success = <condition> return success Pytest treats the return value as a warning, not a failure, so every one of these tests was reported as PASSED regardless of whether Modal worked. The file was effectively a no-op under pytest, masking real Modal terminal regressions. Fix: - Replace each 'return success / return isolated / return requirements_met' with an 'assert <value>, <diagnostic message>'. The diagnostic includes the exit code and error so a CI failure is actionable. - Replace the 'TERMINAL_ENV != modal -> return False' in test_modal_requirements() with pytest.skip(), which is the correct signal when a precondition isn't met. The test is no longer misreported as PASSED when Modal isn't configured; it's cleanly skipped. - Keep the existing imperative main() runner usable (for the 'python tests/integration/test_modal_terminal.py' usage documented in the module docstring). Rewrote it to tolerate AssertionError / pytest skip signals and classify each test as PASSED / FAILED / SKIPPED in the summary.
Collaborator
Contributor
Author
|
Hey @alt-glitch — small context note: this PR was opened ~6 minutes before #13325 (04:48:53Z vs 04:54:55Z), so the timeline runs the other direction here. Happy to defer to @sgaofen's implementation if maintainers prefer it — just wanted to flag the ordering. Whichever lands first is fine. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Six of the tests in
tests/integration/test_modal_terminal.pyused:Pytest treats a returned value as a warning, not a failure. Every one of these tests has been reported as PASSED regardless of whether Modal actually worked — the file was effectively a no-op under pytest, masking real Modal terminal regressions.
Fix:
return success / return isolated / return requirements_metwithassert <value>, <diagnostic message>. The diagnostic includes exit code and error text so a CI failure is actionable.TERMINAL_ENV != modal → return Falseintest_modal_requirements()withpytest.skip(...), which is the correct signal when a precondition isn't met. The test is no longer misreported as PASSED when Modal isn't configured — it's cleanly skipped.main()runner usable (for thepython tests/integration/test_modal_terminal.pyusage the module docstring advertises). Rewrote it to tolerateAssertionError/pytest.skip.Exceptionand classify each test as PASSED / FAILED / SKIPPED in the summary.Related Issue
Fixes #13286
Type of Change
Changes Made
tests/integration/test_modal_terminal.py(+52 / −29): every test now ends withassert ..., <diagnostic>;test_modal_requirementsusespytest.skip();main()rewritten to classify exceptions.How to Test
pytest tests/integration/test_modal_terminal.py -qwithTERMINAL_ENVunset → all tests reported SKIPPED (was: all PASSED silently).TERMINAL_ENV=modal pytest tests/integration/test_modal_terminal.py -q→ real pass/fail based on Modal behavior.python tests/integration/test_modal_terminal.py→ still works as a script runner with PASSED/FAILED/SKIPPED classification.Checklist
Code
fix(tests):)pytest tests/ -qand all tests passDocumentation & Housekeeping
cli-config.yaml.example— N/ACONTRIBUTING.md/AGENTS.md— N/ANot in scope
test_web_tools.pycollects 0 tests) — addressed by a different PR.@pytest.mark.modalfixtures with conftest auto-skip — would be a larger refactor; this PR intentionally stays minimal.Screenshots / Logs
python3 -m py_compile tests/integration/test_modal_terminal.py→ clean.assertcarrying a descriptive message, and thattest_modal_requirements()usespytest.skip()instead ofreturn FalsewhenTERMINAL_ENV != "modal".