Skip to content

fix(runner): don't stumble over directory imports#31915

Closed
Skn0tt wants to merge 8 commits intomicrosoft:mainfrom
Skn0tt:fix-31811
Closed

fix(runner): don't stumble over directory imports#31915
Skn0tt wants to merge 8 commits intomicrosoft:mainfrom
Skn0tt:fix-31811

Conversation

@Skn0tt
Copy link
Copy Markdown
Member

@Skn0tt Skn0tt commented Jul 30, 2024

Closes #31811

Right now, in a monorepo-like setup that involves compilerOptions#paths in tsconfig.json, our esmLoader mechanism can trip up. We're turning a directory path into a file URL, and Node.js doesn't like that. This happens around https://github.com/Skn0tt/playwright/blob/e1aac57dfa6bccabf7deff6c54c3684b753926eb/packages/playwright/src/util.ts#L322-L333, where we purposefully don't try resolving the index file based on package.json.

@Skn0tt Skn0tt requested review from dgozman and mxschmitt July 30, 2024 14:00
@Skn0tt Skn0tt self-assigned this Jul 30, 2024
@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Collaborator

@dgozman dgozman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please put more details into the description? What do we currently resolve to? What should happen instead? Does this only manifest with path mappings in tsconfig.json?

const filename = url.fileURLToPath(context.parentURL);
const resolved = resolveHook(filename, specifier);
if (resolved !== undefined)
if (resolved !== undefined && fs.statSync(resolved).isFile())
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps pass a flag to resolveHook instead to avoid resolving a directory?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea! done in e1aac57

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, looks like that didn't work! #31915 (comment)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, that means we don't understand something. Let's dig until we figure it out.

Copy link
Copy Markdown
Member Author

@Skn0tt Skn0tt Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the test, they use an import style that's not part of the ESM spec, but part of CJS resolution. It feels similar to --experimental-specifier-resolution=node, which never went stable. What's the history behind us supporting this?

Screenshot 2024-07-31 at 11 20 37

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this is about moduleResolution: bundler, and TypeScript resolution is part of the riddle! Fun. #23254

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure why, but 9a873ab seems to have fixed tests.

Copy link
Copy Markdown
Member Author

@Skn0tt Skn0tt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed ☺️

const filename = url.fileURLToPath(context.parentURL);
const resolved = resolveHook(filename, specifier);
if (resolved !== undefined)
if (resolved !== undefined && fs.statSync(resolved).isFile())
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea! done in e1aac57

@github-actions

This comment has been minimized.

@Skn0tt Skn0tt marked this pull request as draft July 30, 2024 18:06
@github-actions

This comment has been minimized.

@Skn0tt Skn0tt marked this pull request as ready for review July 31, 2024 09:55
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@Skn0tt
Copy link
Copy Markdown
Member Author

Skn0tt commented Aug 7, 2024

I felt a little inconfident in this PR because it was one of my earliest changes to the codebase. I've reviewed the entire esmLoader.ts logic now, and believe I have a good understanding of it now.

The important thing here is that while ESM does not allow directory imports, TypeScript does. This only shows with compilerOptions: path: { ... } in the tsconfig, because all other instances would import from node_modules. We explicitly opt out from changing resolution for files in node_modules though (code).

That's why the correct change is to only touch directory resolution when paths are set in tsconfig. I've updated the PR to that, and think we should be good to go. @dgozman could you give the PR another review?

@github-actions

This comment has been minimized.

['', ['.js', '.ts', '.jsx', '.tsx', '.cjs', '.mjs', '.cts', '.mts']],
]);
export function resolveImportSpecifierExtension(resolved: string): string | undefined {
export function resolveImportSpecifierExtension(resolved: string, dontResolveDirectories: boolean): string | undefined {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd reverse the option for easier understanding: resolveDirectories: boolean.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in 8fe370a

}

if (dirExists(resolved)) {
if (dontResolveDirectories)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this check before doing an expensive dirExists() call.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in 8fe370a

const builtins = new Set(Module.builtinModules);

export function resolveHook(filename: string, specifier: string): string | undefined {
export function resolveHook(filename: string, specifier: string, dontResolveDirectories = false): string | undefined {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's call the new argument isModule and make it required. This will expose all the places where we need to think about it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you explain why you'd call it isModule? Because of "module loader"?

The other callsites of resolveHook are inside component testing, and are used to turn Vite's import info into filenames. I think the changes we're making here also apply there, so I don't think we even need a parameter. I've removed it in 8fe370a.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 8, 2024

Test results for "tests 1"

3 fatal errors, not part of any test
10 failed
❌ [playwright-test] › resolver.spec.ts:232:5 › should respect complex path resolver
❌ [playwright-test] › resolver.spec.ts:572:5 › should import packages with non-index main script through path resolver
❌ [playwright-test] › resolver.spec.ts:232:5 › should respect complex path resolver
❌ [playwright-test] › resolver.spec.ts:572:5 › should import packages with non-index main script through path resolver
❌ [playwright-test] › resolver.spec.ts:232:5 › should respect complex path resolver
❌ [playwright-test] › resolver.spec.ts:572:5 › should import packages with non-index main script through path resolver
❌ [playwright-test] › resolver.spec.ts:232:5 › should respect complex path resolver
❌ [playwright-test] › resolver.spec.ts:572:5 › should import packages with non-index main script through path resolver
❌ [playwright-test] › resolver.spec.ts:232:5 › should respect complex path resolver
❌ [playwright-test] › resolver.spec.ts:572:5 › should import packages with non-index main script through path resolver

3 flaky ⚠️ [firefox-page] › page/frame-goto.spec.ts:46:3 › should continue after client redirect
⚠️ [firefox-page] › page/page-goto.spec.ts:182:3 › should properly cancel Cross-Origin-Opener-Policy navigation
⚠️ [playwright-test] › ui-mode-test-watch.spec.ts:145:5 › should watch all

29501 passed, 710 skipped
✔️✔️✔️

Merge workflow run.

@Skn0tt
Copy link
Copy Markdown
Member Author

Skn0tt commented Aug 8, 2024

superceded by #32078

@Skn0tt Skn0tt closed this Aug 8, 2024
Skn0tt added a commit that referenced this pull request Aug 12, 2024
…x.js` and `package.json` through path mapping (#32078)

Supercedes #31915, closes
#31811.

When TypeScript resolves a specifier via path mapping, it does not
interpret `package.json`. If path mapping resolves to a directory, it
only looks at the `index.js` file in that directory if it's in CommonJS
mode.

We need to mirror this in our `esmLoader.ts`.

---------

Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Regression]: ESM Directory Import is not working anymore with Playwright's ESM Loader activated

2 participants