@zkochan I have two questions according to your comment:

we'll have to probably detect if the workspace feature is used and if so, then run install in the root of the repository before taking the subdirectory.

The "workspace" feature in NPM, Yarn and PNPM is different. PNPM uses WORKSPACE_MANIFEST_FILENAME (pnpm-workspace.yaml) while the others use workspaces field in package.json. Should I support both two scenarios?

It seems the only work is to add logic to the packageShouldBeBuilt function; if it returns true, PNPM will run ${pm} install then.

git+https://host/user/repo#next#path:<relative path>

I found that Yarn's "workspace clone" is actually placed on the docs Git Protocol, which gives a syntax that uses & to split multiple params:

git@github.com:yarnpkg/berry.git#workspace=@yarnpkg/shell&tag=@yarnpkg/shell/2.1.0

Should I change the separator to &?

The "workspace" feature in NPM, Yarn and PNPM is different. PNPM uses WORKSPACE_MANIFEST_FILENAME (pnpm-workspace.yaml) while the others use workspaces field in package.json. Should I support both two scenarios?

It seems the only work is to add logic to the packageShouldBeBuilt function; if it returns true, PNPM will run ${pm} install then.

The current logic might already work for the monorepo scenario. pnpm checks what lockfile is used and runs the right package manager.

which gives a syntax that uses & to split multiple params

I'm fine with this approach just use path instead of workspace

The current logic might already work for the monorepo scenario.

The current logic does not handle the pnpm-workspace.yaml or workspaces field - if there's no scripts, prepare, index.js etc., the preparePackage just returns false without executing any command.

I'll add a check for this that only runs when path is present.

@zkochan I've finished all changes; please take a look~

add changesets.

Also, this is a new feature, so I'd like more members to review it.

What to do with packages that have the workspace:* in dependencies? For instance, I want to install @pnpm/fetch directly from the pnpm repository. It has these dependencies:

Installation will fail because workspace:* can be resolved only inside a workspace.

I guess the "right" solution would be to convert those dependencies to something like:

 "@pnpm/core-loggers": "github:pnpm/pnpm#main&path:packages/core-loggers", 
 "@pnpm/fetching-types": "github:pnpm/pnpm#main&path:network/fetching-types", 

Using workspace:* is currently impossible even without specifying a path. There's only a .bin folder inside the node_modules of each subpackage because soft links become invalid when moving files.

The scenario may need to be supported, but maybe not by this PR?

Currently, pnpm works fine only for the following scenarios (without workspace:*):

• The repo has a prepublish script and will bundle a dist file during prepublish to let users import.
• Users want to use the source code (maybe in src) in this repo, and there are dependencies that will be used by the source code.
• Users want to use the source code (maybe in src) in this repo, and there is no dependencies at all.

After this PR, pnpm will also work fine in these scenarios if users specify path. I think maybe it's enough.

I've added support for tarball fetcher, and changed pkgDir param of preparePackage to be required to prevent potential errors caused by missing the param in future.

It still doesn't work as far as I can tell. Will check later why. Also, it is stored currently in the store by git URL, so it will work only for the first package from the given git repo, the next one will be broken.

It should be okay to store only by Git URL - just 1 item in packages and multiple items in dependencies.

But I found it strange that pnpm somehow uses both tarball fetcher and git fetcher even if the Git URLs are the same. Please see the following package.json and the generated pnpm-lock.yaml:

{
  "name": "test",
  "version": "1.0.0",
  "dependencies": {
    "simple-react-app": "github:RexSkz/test-git-subfolder-fetch.git#path:/packages/simple-react-app",
    "simple-express-server": "github:RexSkz/test-git-subfolder-fetch.git#path:/packages/simple-express-server"
  }
}

dependencies:
  simple-express-server:
    specifier: github:RexSkz/test-git-subfolder-fetch.git#path:/packages/simple-express-server
    version: git@github.com+RexSkz/test-git-subfolder-fetch/2b42a57a945f19f8ffab8ecbd2021fdc2c58ee22
  simple-react-app:
    specifier: github:RexSkz/test-git-subfolder-fetch.git#path:/packages/simple-react-app
    version: github.com/RexSkz/test-git-subfolder-fetch/2b42a57a945f19f8ffab8ecbd2021fdc2c58ee22

packages:

  git@github.com+RexSkz/test-git-subfolder-fetch/2b42a57a945f19f8ffab8ecbd2021fdc2c58ee22:
    resolution: {commit: 2b42a57a945f19f8ffab8ecbd2021fdc2c58ee22, path: /packages/simple-express-server, repo: git@github.com:RexSkz/test-git-subfolder-fetch.git, type: git}
    name: '@my-namespace/simple-express-server'
    version: 1.0.0
    dependencies:
      express: 4.18.2
      lodash: 4.17.21
    transitivePeerDependencies:
      - supports-color
    dev: false

  github.com/RexSkz/test-git-subfolder-fetch/2b42a57a945f19f8ffab8ecbd2021fdc2c58ee22:
    resolution: {tarball: https://codeload.github.com/RexSkz/test-git-subfolder-fetch/tar.gz/2b42a57a945f19f8ffab8ecbd2021fdc2c58ee22}
    name: monorepo-example
    version: 1.0.0
    dev: false

The simple-express-server is installed correctly, but it seems that pnpm resolves simple-react-app to a tarball URL, and the path param is lost.

I will look into it only after I finish with #7407

TODO:

• add an integration test to @pnpm/core. The test should install two packages from the same monorepo
• consider changing the way index files for git-hosted dependencies are saved in the store. Right now it will be increasingly long, which will cause issues on Windows. Maybe use a hash instead?

Hi, Is it going to be a part of 8.x?

It seems a part of 9.x. https://github.com/pnpm/pnpm/releases/tag/v9.0.0-alpha.1

What to do with packages that have the workspace:* in dependencies? For instance, I want to install @pnpm/fetch directly from the pnpm repository. It has these dependencies:

pnpm/network/fetch/package.json

Lines 36 to 37 in 46990ec

	"@pnpm/core-loggers": "workspace:*",
	"@pnpm/fetching-types": "workspace:*",

Installation will fail because workspace:* can be resolved only inside a workspace.

I guess the "right" solution would be to convert those dependencies to something like:

 "@pnpm/core-loggers": "github:pnpm/pnpm#main&path:packages/core-loggers", 
 "@pnpm/fetching-types": "github:pnpm/pnpm#main&path:network/fetching-types", 

The capability to install packages with workspace:* dependencies is fundamentally important for enabling efficient cross-project referencing. In my experience of maintaining multiple monorepos, the current workflow of propagating changes exclusively through new package releases is both painfully frustrating and inefficient 😭 I truly hope we can find a resolution to this issue in the near future, as it would significantly enhance our development processes 🚀

Hi, Is it going to be a part of 8.x?

No, it will only be part of v9. v8 will probably have only bug fixes from now on.

The capability to install packages with workspace:* dependencies is fundamentally important for enabling efficient cross-project referencing. In my experience of maintaining multiple monorepos, the current workflow of propagating changes exclusively through new package releases is both painfully frustrating and inefficient 😭 I truly hope we can find a resolution to this issue in the near future, as it would significantly enhance our development processes 🚀

It is not an easy task. It isn't high in my own priority list.

In general, I don't think installing from git is a good idea although I do see that many people do it.

I would recommend looking into bit.dev as an alternative to monorepo(s) usage. With bit every component is independent but can be developed together in a singe workspace. So, you don't need to maintain monorepos. When you want to change some components, you just import them to a workspace. When the changes are done, you export them. This workflow is much better than monorepos, especially than multiple monorepos.