Skip to content

coverage: do not run test in strict mode when it doesn't contain onlyStrict and module flags #10057

@Dunqing

Description

@Dunqing

// Unless configured otherwise (via the noStrict, onlyStrict, module, or raw flags),
// each test must be executed twice: once in ECMAScript's non-strict mode, and again in ECMAScript's strict mode.
// To run in strict mode, the test contents must be modified prior to execution--
// a "use strict" directive must be inserted as the initial character sequence of the file
// https://github.com/tc39/test262/blob/05c45a4c430ab6fee3e0c7f0d47d8a30d8876a6d/INTERPRETING.md#strict-mode
fn run(&mut self) {
let flags = &self.meta.flags;
let source_type = SourceType::cjs();
self.result = if flags.contains(&TestFlag::OnlyStrict) {
self.always_strict = true;
self.execute(source_type)
} else if flags.contains(&TestFlag::Module) {
self.execute(source_type.with_module(true))
} else if flags.contains(&TestFlag::NoStrict) || flags.contains(&TestFlag::Raw) {
self.execute(source_type)
} else {
self.always_strict = true;
let res = self.execute(source_type);
if matches!(res, TestResult::Passed) {
self.always_strict = false;
self.execute(source_type)
} else {
res
}
};
}

The test262 documentation said

Strict Mode

Unless configured otherwise (via the noStrict, onlyStrict, module, or
raw flags), each test must be executed twice: once in ECMAScript's non-strict
mode, and again in ECMAScript's strict mode. To run in strict mode, the test
contents must be modified prior to execution--a "use strict"
directive

must be inserted as the initial character sequence of the file, followed by a
semicolon (;) and newline character (\n):

"use strict";

This must precede any additional text modifications described by test metadata.

I think this is incorrect because test262 has many tests that don't contain any strict-related flags and are only valid in non-strict mode.

For example:
https://github.com/tc39/test262/blob/bc5c14176e2b11a78859571eb693f028c8822458/test/language/expressions/dynamic-import/syntax/valid/nested-if-script-code-valid.js

// This file was procedurally generated from the following sources:
// - src/dynamic-import/script-code-valid.case
// - src/dynamic-import/syntax/valid/nested-if.template
/*---
description: import() can be used in script code (nested if syntax)
esid: sec-import-call-runtime-semantics-evaluation
features: [dynamic-import]
flags: [generated]
info: |
    ImportCall :
        import( AssignmentExpression )

    1. Let referencingScriptOrModule be ! GetActiveScriptOrModule().
    2. Assert: referencingScriptOrModule is a Script Record or Module Record (i.e. is not null).
    3. Let argRef be the result of evaluating AssignmentExpression.
    4. Let specifier be ? GetValue(argRef).
    5. Let promiseCapability be ! NewPromiseCapability(%Promise%).
    6. Let specifierString be ToString(specifier).
    7. IfAbruptRejectPromise(specifierString, promiseCapability).
    8. Perform ! HostImportModuleDynamically(referencingScriptOrModule, specifierString, promiseCapability).
    9. Return promiseCapability.[[Promise]].

---*/
// This is still valid in script code, and should not be valid for module code
// https://tc39.github.io/ecma262/#sec-scripts-static-semantics-lexicallydeclarednames
var smoosh; function smoosh() {}


if (true) {
  import('./empty_FIXTURE.js');
}

If this test needs to execute in strict mode, then we will get a syntax error because the smoosh has already been declared. But it allows executing in non-strict mode.

Aside from this reason, these #10050 #10051 #10052 #10053 #10055 #10056 issues are related to this problem because they run in strict mode and will get an expected error.

Conclusion

The strict mode-related documentation in test262 is probably wrong, we shouldn't execute tests in strict mode when it doesn't contain onlyStrict and module flags.

Metadata

Metadata

Assignees

Labels

C-testCategory - Testing. Code is missing test cases, or a PR is adding them

Type

Priority

None yet

Effort

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions