Skip to content

Bug: tests/integration/test_modal_terminal.py returns booleans so logical failures still pass under pytest #13286

@NewTurn2017

Description

@NewTurn2017

Summary

tests/integration/test_modal_terminal.py defines pytest-style test_* functions, but each test returns True/False instead of asserting.

Pytest treats non-None returns as warnings (PytestReturnNotNoneWarning), not failures, so logically failing checks can still show up as passing tests.

Affected file

  • tests/integration/test_modal_terminal.py:58-237

Why this is a bug

This hides Modal/backend regressions while giving a false green signal in CI/manual pytest runs.

Reproduction

cd /Users/genie/.hermes/hermes-agent
source venv/bin/activate
TERMINAL_ENV=local pytest -q -m integration -o addopts='' tests/integration/test_modal_terminal.py -k test_modal_requirements

Observed output:

.                                                                        [100%]
PytestReturnNotNoneWarning: Test functions should return None, but ... returned <class 'bool'>
1 passed, 5 deselected, 1 warning

Inside the test, the current environment clearly does not satisfy the requirement:

  • tests/integration/test_modal_terminal.py:76-79 prints a warning and return False

Expected behavior

A failed requirement check should fail the pytest test.

Actual behavior

The test returns False, pytest emits a warning, and the test still passes.

Suggested investigation direction

Convert these functions to normal pytest assertions (or rename them away from test_* and keep them as manual scripts if that was the intent).

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — cosmetic, nice to havebackend/modalModal.com cloud executiontool/terminalTerminal execution and process managementtype/testTest coverage or test infrastructure

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions