Skip to content

op-acceptance-tests output does not handle package-level test declarations #14803

@teddyknox

Description

@teddyknox

Bug Description

Unfortunately there is a bug in just acceptance-test that causes package level test configurations to not appear in the test output. Configurations like the following do not seem to works:

      - package: github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/interop

And produce output like the following (see how the total column doesn't add up):

t=2025-03-12T01:16:24+0000 lvl=info msg=NewTestRunner() targetGate=localnet workDir=/go/src/github.com/ethereum-optimism/optimism
t=2025-03-12T01:16:24+0000 lvl=info msg="nat.New: created registry and test runner"
t=2025-03-12T01:16:24+0000 lvl=info msg="Starting op-acceptor"
t=2025-03-12T01:16:24+0000 lvl=info msg="Running all tests[n.runner.RunAllTests()]..."
t=2025-03-12T01:16:24+0000 lvl=info msg="Running test" validator=TestFindRPCEndpoints
t=2025-03-12T01:16:46+0000 lvl=info msg="Running test" validator=TestSystemWrapETH
t=2025-03-12T01:17:06+0000 lvl=info msg="Running test" validator=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/interop
t=2025-03-12T01:17:07+0000 lvl=info msg="runAllTestsInPackage() found tests" package=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/interop count=5
t=2025-03-12T01:17:11+0000 lvl=info msg="Running test" validator=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/isthmus
t=2025-03-12T01:17:30+0000 lvl=info msg="runAllTestsInPackage() found tests" package=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/isthmus count=4
t=2025-03-12T01:17:35+0000 lvl=info msg="Running test" validator=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/fjord
t=2025-03-12T01:17:38+0000 lvl=info msg="runAllTestsInPackage() found tests" package=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/fjord count=2
t=2025-03-12T01:17:40+0000 lvl=info msg="Printing results..."
 Acceptance Testing Results (76.2s)                                                       
 TYPE   ID                        DURATION  TESTS  PASSED  FAILED  SKIPPED  STATUS  ERROR 
 Gate   localnet                     76.2s      -       4       0        1  ✓ pass        
 Test   ├── TestFindRPCEndpoints     22.4s      1       1       0        0  ✓ pass  <nil> 
        ├── TestSystemWrapETH        19.7s      1       0       0        1  - skip  <nil> 
        └──                           5.0s      1       1       0        0  ✓ pass  <nil> 
 TOTAL                                76.2S     5       4       0        1  ✓ PASS        

Steps to Reproduce

Run the test configuration as set in #14802 using just acceptance-test.

Expected behavior

We expect there to be output table lines for each test package.


⚠️ Notice: Issues that do not include the following sections will be subject to closure:

  • Bug Description
  • Steps to Reproduce
  • Environment Information

Please ensure all required sections are filled out accurately to expedite the debugging process and improve issue resolution efficiency.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions