-
Notifications
You must be signed in to change notification settings - Fork 3.9k
op-acceptance-tests output does not handle package-level test declarations #14803
Copy link
Copy link
Closed
Closed
Copy link
Labels
Description
Bug Description
Unfortunately there is a bug in just acceptance-test that causes package level test configurations to not appear in the test output. Configurations like the following do not seem to works:
- package: github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/interop
And produce output like the following (see how the total column doesn't add up):
t=2025-03-12T01:16:24+0000 lvl=info msg=NewTestRunner() targetGate=localnet workDir=/go/src/github.com/ethereum-optimism/optimism
t=2025-03-12T01:16:24+0000 lvl=info msg="nat.New: created registry and test runner"
t=2025-03-12T01:16:24+0000 lvl=info msg="Starting op-acceptor"
t=2025-03-12T01:16:24+0000 lvl=info msg="Running all tests[n.runner.RunAllTests()]..."
t=2025-03-12T01:16:24+0000 lvl=info msg="Running test" validator=TestFindRPCEndpoints
t=2025-03-12T01:16:46+0000 lvl=info msg="Running test" validator=TestSystemWrapETH
t=2025-03-12T01:17:06+0000 lvl=info msg="Running test" validator=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/interop
t=2025-03-12T01:17:07+0000 lvl=info msg="runAllTestsInPackage() found tests" package=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/interop count=5
t=2025-03-12T01:17:11+0000 lvl=info msg="Running test" validator=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/isthmus
t=2025-03-12T01:17:30+0000 lvl=info msg="runAllTestsInPackage() found tests" package=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/isthmus count=4
t=2025-03-12T01:17:35+0000 lvl=info msg="Running test" validator=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/fjord
t=2025-03-12T01:17:38+0000 lvl=info msg="runAllTestsInPackage() found tests" package=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/fjord count=2
t=2025-03-12T01:17:40+0000 lvl=info msg="Printing results..."
Acceptance Testing Results (76.2s)
TYPE ID DURATION TESTS PASSED FAILED SKIPPED STATUS ERROR
Gate localnet 76.2s - 4 0 1 ✓ pass
Test ├── TestFindRPCEndpoints 22.4s 1 1 0 0 ✓ pass <nil>
├── TestSystemWrapETH 19.7s 1 0 0 1 - skip <nil>
└── 5.0s 1 1 0 0 ✓ pass <nil>
TOTAL 76.2S 5 4 0 1 ✓ PASS
Steps to Reproduce
Run the test configuration as set in #14802 using just acceptance-test.
Expected behavior
We expect there to be output table lines for each test package.
- Bug Description
- Steps to Reproduce
- Environment Information
Please ensure all required sections are filled out accurately to expedite the debugging process and improve issue resolution efficiency.
Reactions are currently unavailable