Skip to content

refactor: store workspace manifest in @pnpm/config for reuse#8213

Merged
zkochan merged 3 commits intomainfrom
gluxon/hoist-manifest-read-to-config
Jun 16, 2024
Merged

refactor: store workspace manifest in @pnpm/config for reuse#8213
zkochan merged 3 commits intomainfrom
gluxon/hoist-manifest-read-to-config

Conversation

@gluxon
Copy link
Member

@gluxon gluxon commented Jun 16, 2024

PR Stack

Changes

The pnpm-workspace.yaml file is currently read whenever findWorkspacePackages is called and then discarded.

Following recent discussions in other PRs, let's store pnpm-workspace.yaml in @pnpm/config for longer-term reuse and pass the packages field to findWorkspacePackages when possible.

approach 1 is the right one. Maybe we can read it in @pnpm/config. We already read the root manifest there too. #8020 (comment)

This sounds good to me. I guess you could put patterns to the options. #8120 (comment)

This makes it easier to reuse the contents of pnpm-workspace.yaml for pnpm catalogs.

Risks

This PR is mostly a no-op refactor, but it does change the timing of when pnpm-workspace.yaml is read to be a bit earlier than it was before.

* directory and contents of the workspace manifest together without null
* checks.
*/
dir: string
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it possible to just check whether opts.workspaceManifest is set and have both opts.workspaceManifest.dir and opts.workspaceManifest.contents be typed as non-null.

if (opts.workspaceManifest) {
  await findWorkspacePackages(opts.workspaceManifest.dir, {
    ...opts,
    patterns: opts.workspaceManifest.contents.packages
  })
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zkochan Adding a workspacePackagePatterns field to the top-level of Config makes it harder to write the code above.

We have to check that both workspacePackagePatterns and workspaceDir are non-null, but that's a bit of a code smell because they're supposed to either be both defined, or both undefined.

if (opts.workspaceDir != null && opts.workspacePackagePatterns != null) {
  await findWorkspacePackages(opts.workspaceDir, {
    ...opts,
    patterns: opts.workspacePackagePatterns
  })
}

This is a problem for the next PR in #8214.

Maybe we can do something like this in the Config object?

interface Config {
  workspace?: {
    dir: string;
    packagePatterns: string;

    // other existing fields might also make sense here
    hoistWorkspacePackages?: boolean;
  }
}

const allProjects = opts.allProjects ?? await findWorkspacePackages(opts.workspaceDir, opts)
const allProjects = opts.allProjects ?? await findWorkspacePackages(opts.workspaceDir, {
...opts,
patterns: opts.workspaceManifest?.contents.packages,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the comment above, the optional chain here isn't necessary if we check opts.workspaceManifest is non-null instead of opts.workspaceDir.

That's a potentially larger refactor though. Tackling this in a different PR.

useRunningStoreServer?: boolean
workspaceConcurrency: number
workspaceDir?: string
workspaceManifest?: {
Copy link
Member

@zkochan zkochan Jun 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we just need to add the setting from workspace manifest to the rest of the settings. Why should the users of the config be aware where the settings are coming from? Just add workspacePatterns to the Config object.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I was attempting to follow the rootProjectManifest option below, which grouped settings under where it comes from. It sounds like that wasn't a good example to follow?

What I do like about the existing setup here is that it better models what can happen at runtime. Grouping options together like this makes it possible to declare many options that are all non-null/defined if any of them are non-null. This means we don't have to handle non-existent states at compile time. In this case, we know both the workspace dir and patterns are non-null if either are non-null.

In a future PR, we'd also know the patterns and dir if catalogs are defined. And that intuitively makes sense since catalogs are a feature that only work in a workspace.

pnpm/src/main.ts Outdated
engineStrict: config.engineStrict,
nodeVersion: config.nodeVersion ?? config.useNodeVersion,
patterns: cliOptions['workspace-packages'],
patterns: cliOptions['workspace-packages'] ?? config.workspaceManifest?.contents.packages,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a new setting to the config object that will prioritize the workspace-packages cli flag over the. packages from workspace manifest. Here you'll just need to do

Suggested change
patterns: cliOptions['workspace-packages'] ?? config.workspaceManifest?.contents.packages,
patterns: config.workspacePackagePatterns,

Copy link
Member Author

@gluxon gluxon Jun 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can add the new config.workspacePackagePatterns field, but do you mind if I fold the cliOptions['workspace-packages'] in a separate PR?

I just want to do a bit more testing to ensure there's no behavior changes. Today, cliOptions['workspace-packages'] is only read here in this if (cliOptions['recursive']) { block. I want to make sure things don't unexpectedly break if that CLI option takes priority always elsewhere we use patterns.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is fine if it will take priority always.

Copy link
Member Author

@gluxon gluxon Jun 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I do this in another PR after this merges? I wanted this PR to just be a refactor without behavior changes. The change above might subtly change how pnpm behaves.

Copy link
Member Author

@gluxon gluxon Jun 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand we want cliOptions['workspace-packages'] to take precedence always, but the problem is it doesn't today in all cases. Fixing that might introduce other bugs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't matter. That flag was added for one specific user (who seems like moved to bun) and we don't even have the flag documented on the website.

You can do a separate PR but nobody will notice the difference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha. I was probably a bit too afraid of risk in this case. I'm glad to hear nobody will notice either way.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making this change in #8215.

@gluxon gluxon force-pushed the gluxon/hoist-manifest-read-to-config branch from a9d8daa to 9136a66 Compare June 16, 2024 19:14
@gluxon
Copy link
Member Author

gluxon commented Jun 16, 2024

Updating this PR to store workspacePackagePatterns into the Config interface instead of workspaceManifest per feedback.

@zkochan zkochan merged commit 04b8363 into main Jun 16, 2024
@zkochan zkochan deleted the gluxon/hoist-manifest-read-to-config branch June 16, 2024 23:26
@gluxon gluxon mentioned this pull request Jun 26, 2024
18 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants