Skip to content

(wdio-config): check existence of ts-node/esm/transpile-only.mjs & fix a type issue#12436

Merged
christian-bromann merged 4 commits intowebdriverio:mainfrom
johnp:ts-node-compat
Mar 9, 2024
Merged

(wdio-config): check existence of ts-node/esm/transpile-only.mjs & fix a type issue#12436
christian-bromann merged 4 commits intowebdriverio:mainfrom
johnp:ts-node-compat

Conversation

@johnp
Copy link
Contributor

@johnp johnp commented Mar 6, 2024

Proposed changes

Fixes two issues behind #12434. The first fix is to check for the existence of the ts-node/esm/transpile-only.mjs loader before trying to use it, as it does not exist in ts-node versions <9 (in our case pulled in by a transitive dependency). The 2nd fix is a type issue with the argument autoCompileConfig.autoCompile=false which ends up as the string "false". I added a conversion step to take care of this. Note that this CLI argument is already documented in wdio --help.

Types of changes

  • Polish (an improvement for an existing feature)
  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update (improvements to the project's docs)
  • Specification changes (updates to WebDriver command specifications)
  • Internal updates (everything related to internal scripts, governance documentation and CI files)

Checklist

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works
  • I have added the necessary documentation (if appropriate)
  • I have added proper type definitions for new commands (if appropriate)

Backport Request

Further comments

Reviewers: @webdriverio/project-committers

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Mar 6, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

@johnp johnp force-pushed the ts-node-compat branch 2 times, most recently from 53feeb6 to 8c332de Compare March 6, 2024 00:56
@johnp johnp marked this pull request as ready for review March 6, 2024 01:11
@johnp johnp marked this pull request as draft March 6, 2024 01:15
@erwinheitzman
Copy link
Member

Hi @johnp currently the access is checked but nothing is done with the result which doesn't seem right.

Usually you see something like:

try {
  await access('path')
} catch (error) {
  // do something
}

Regarding the autocompileopts, I am planning to remove it and instead recommend use of the environment variables as shown in our docs here: https://webdriver.io/docs/typescript as this is how ts-node expects things to be used. It also prevents us from adding magic/logic that doesn't have to be maintained and isn't needed.

@johnp
Copy link
Contributor Author

johnp commented Mar 6, 2024

Hi @erwinheitzman,

the access call is already in a try-catch clause which catches the ENOENT error and then returns false. I have a small new vitest locally which verifies that this indeed works.

I don't want to use ts-node since it breaks all our tests which work just fine with babel and wdio v7. AFAIK there is no ts-node environment variable which disables the ts-node loader(s) altogether.

I have a hacky workaround by registering babel myself and passing --autoCompileOpts.autoCompile= as an argument, which ends up as an empty string, which is falsy, so loadAutoCompilers immediately returns, but that is explicitly in contradiction to the documentation in wdio --help. I would very much like to avoid having add that argument to the hundreds of wdio calls we have in our repos, so I would prefer to also have the "automatic" existence check for ts-node/esm/transpile-only.mjs.

Right now I'm only struggling with this test:

it('should just continue without initiation when autoCompile:false', async () => {
process.env.THROW_BABEL_REGISTER = '1'
const configParserBuilder = ConfigParserBuilder
.withBaseDir(FIXTURES_PATH, FIXTURES_CONF_RDC, {
autoCompileOpts: {
autoCompile: false
}
})
.withBabelModule()
.withFiles([
...MockedFileSystem_OnlyLoadingConfig(process.cwd(), FIXTURES_CONF_RDC)
])
const { requireMock } = configParserBuilder.getMocks().modules.getMocks()
const configParser = configParserBuilder.build()
await configParser.initialize()
expect(requireMock).not.toHaveBeenCalled()
})

It always succeeds, even if I set autoCompile: true. It should clearly fail in this case but it doesn't. Once that works I can add a simple near-duplicate that just uses the string 'false' to verify that the documented CLI argument also works again:

            it('should just continue without initiation with --autoCompileOpts.autoCompile=false', async () => {
                vi.stubEnv('THROW_BABEL_REGISTER', '1')
                const configParserBuilder = ConfigParserBuilder
                    .withBaseDir(FIXTURES_PATH, FIXTURES_CONF_RDC, {
                        autoCompileOpts: {
                            autoCompile: 'false'
                        }
                    })
                    .withBabelModule()
                    .withFiles([
                        ...MockedFileSystem_OnlyLoadingConfig(process.cwd(), FIXTURES_CONF_RDC)
                    ])
                const { requireMock } = configParserBuilder.getMocks().modules.getMocks()
                const configParser = configParserBuilder.build()
                await configParser.initialize()
                expect(requireMock).not.toHaveBeenCalled()
            })

I know you're working on some stuff with vite for v9, so if you'd prefer that I could also only work on / submit the other PR against v8 and close this one against main.

@erwinheitzman
Copy link
Member

@johnp there's a few things I want to make sure that are clear.

  1. you mention v7, are you using v7 or v8 right now? and if you do use v7, why are you updating v8?
  2. you mention you use Babel, if you use this and not Typescript, it might be a configuration issue rather than an issue with WebdriverIO's logic

Could you share more information about your project so that I can exclude some possibilities?

@johnp
Copy link
Contributor Author

johnp commented Mar 6, 2024

@erwinheitzman We are on wdio v7 and I'd like to get us on wdio v8. Primarily to just get us up2date and also because of Chromium/chromedriver issues we've been having, which we hope to solve by using Chrome for Testing, which wdio v8 natively provides (and always keeps in sync with the chromedriver version; great feature :) ).

In fact, with the hacky workaround I described, our existing tests run just fine on wdio v8 & babel.

I don't think there's any deeper problem in WebdriverIO's logic. I've tried using ts-node v10 and the v11 beta and both just cause a lot of cjs/esm import errors on our tests which babel happily complies with. We'd therefore like to simply not use ts-node. Unfortunately, we have a dependency which pulls in ts-node v8, which means that this line

await resolve('ts-node', import.meta.url)

doesn't throw an error, but the NODE_OPTION --loader ts-node/esm/transpile-only, which WebdriverIO then injects, does. (because ts-node v8 doesn't have that file). That's why I've added the check for the `ts-node/esm/transpile-only.mjs.

I've uploaded a minimized example project here: https://github.com/johnp/wdio-ts-node-8-bug (edit: just made public)

@christian-bromann
Copy link
Member

I just want to point out that we have #11878 planned for v9. I wonder if this might help us resolve some of these issues.

@johnp
Copy link
Contributor Author

johnp commented Mar 7, 2024

@christian-bromann Yes, tsx/esbuild might work better, though I would still hope there to be an option to not forcibly load it. @delagen, in that issue, seems to have encountered that same problem as I have here.

@delagen Maybe you can use the workaround I found here (passing the argument --autoCompileOpts.autoCompile=) to avoid having to edit your lock file by hand for now.

@johnp
Copy link
Contributor Author

johnp commented Mar 7, 2024

Regarding my struggle with the one test here #12436 (comment), I was simply confused by the interplay between the environment variables and the mocked modules. Everything makes sense now.

Given that this will go away soon anyways, I've just added two small tests (primarily useful to prevent regressions on the v8 branch).

@johnp johnp marked this pull request as ready for review March 7, 2024 23:32
@erwinheitzman
Copy link
Member

erwinheitzman commented Mar 8, 2024

Can you explain why you need this when you can set autoCompile to false in your config (you can also remove the autoCompileConfig which should achieve what you want)?

@johnp
Copy link
Contributor Author

johnp commented Mar 8, 2024

@erwinheitzman Nothing set in the config can prevent WebdriverIO from trying to use ts-loader (or babel) to parse the config file (even if the config file is a .js file). Chicken & egg problem. See here:

async initialize(object: MergeConfig = {}) {
/**
* only run auto compile functionality once but allow the config parse to be initialized
* multiple times, e.g. when used with the packages/wdio-cli/src/watcher.ts
*/
if (!this.#isInitialised) {
await loadAutoCompilers(this._config.autoCompileOpts!, this._moduleRequireService)
await this.addConfigFile(this.#configFilePath)

The loadAutoCompilers function is called before the config is even read.

In my example repo the first package.json script reproduces that this is happening.

@erwinheitzman
Copy link
Member

@johnp good points. looks good to me. what do you think @christian-bromann ? What I do think is that perhaps we should merge this only to v8

Copy link
Member

@christian-bromann christian-bromann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Let's move forward and merge this for v9 too in case we decide not to migrate to tsx.

Thanks @johnp

@christian-bromann christian-bromann added the PR: Bug Fix 🐛 PRs that contain bug fixes label Mar 9, 2024
@christian-bromann christian-bromann merged commit bd1d047 into webdriverio:main Mar 9, 2024
@wdio-bot
Copy link
Contributor

wdio-bot commented Mar 9, 2024

Hey johnp 👋

Thank you for your contribution to WebdriverIO! Your pull request has been marked as an "Expensable" contribution. We've sent you an email with further instructions on how to claim your expenses from our development fund. Please make sure to check your spam folder as well. If you have any questions, feel free to reach out to us at expense@webdriver.io or in the contributing channel on Discord.

We are looking forward to more contributions from you in the future 🙌

Have a nice day,
The WebdriverIO Team 🤖

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Expensable $35 💸 PR: Bug Fix 🐛 PRs that contain bug fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants