Skip to content

run-test262 does not always report errors #970

@bnoordhuis

Description

@bnoordhuis
$ build/run-test262 -c test262.conf -d test262/test/built-ins/AsyncFromSyncIteratorPrototype/throw
test262.conf:38: ignoring testdir=test262/test
Result: 6/15 errors

$ echo $?
0

Vs. (note the trailing slash)

$ build/run-test262 -c test262.conf -d test262/test/built-ins/AsyncFromSyncIteratorPrototype/throw/
test262.conf:38: ignoring testdir=test262/test
test262/test/built-ins/AsyncFromSyncIteratorPrototype/throw//iterator-result-rejected-promise-close.js:74: unexpected error: TypeError: $DONE() not called
test262/test/built-ins/AsyncFromSyncIteratorPrototype/throw//throw-undefined-get-return-undefined.js:64: unexpected error: TypeError: $DONE() not called
test262/test/built-ins/AsyncFromSyncIteratorPrototype/throw//throw-undefined-return-object.js:66: unexpected error: TypeError: $DONE() not called
test262/test/built-ins/AsyncFromSyncIteratorPrototype/throw//throw-undefined-return-not-object.js:72: unexpected error: TypeError: $DONE() not called
test262/test/built-ins/AsyncFromSyncIteratorPrototype/throw//throw-undefined-poisoned-return.js:68: unexpected error: TypeError: $DONE() not called
test262/test/built-ins/AsyncFromSyncIteratorPrototype/throw//throw-result-poisoned-wrapper.js:81: unexpected error: TypeError: $DONE() not called
Result: 6/15 errors, 6 new

$ echo $?
1

I suspect it causes run-test262 (both locally and on CI) to under-report test failures. edit: it's the other way around. The failures are listed in test262_errors.txt and expected. The trailing slash makes them get over-reported because of a bad check.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions