I have updated the branch to master, resolving conflicts and updating some tests. I have also reviewed the branch.
Overall, it works well and keeps compatibility with older versions because it uses <project_path>.nextflow to store the bare repo and commits. Apart from the caveats commented in the PR, I have found the following ones:

• Unnecessary full clones: For every commitId folder, it is reusing the download method that is doing a full clone. There is no need to do a full clone, it should just include the branch to remove. I think it is because the default branch was difficult to get. However, in bare repos, it is store in the HEAD file and the reference is easily obtained by getting the HEAD ref of the bare repo.

• Use commit clones to get the list of Branches and Tags. Related to the previous comment, the AssetManager.localPath property is set to <project_path>.nextflow/commits/<commitId> and the git repository used to get the list of branches and tags is using this folder. This information is also in the bare repo. No reason to get from localPath.

• Drop command has an issue with commits that are not in the revisions map file. For instance, an update of a branch that has been pulled before creates a kind of orphan folder with the previous commit id. When I try to remove it using the commit Id, drop fails.

• Drop with -all option is iterating the revisions map, creating assets manager for each revision and removing one by one. Finally, it removes the whole project folder, so this iteration is unnecessary, we can direcly remote the directory

TODO:

• Remove full clones in commit folders
• Get HEAD, branches and tags from bare repo
• Fix drop issues
• Remove default branch duplication
• fix tag problems
• Address concurrent pulls of the same revision (file lock?)

Fixed most of TODOs:

• The tag issue mentioned Enable multiple git revisions (Jun 2024) : final, using commit ID in repo localPath #5089 (comment) is produced by git.repository.nameRev method that returns this strange <tag_name>~<number> when a commit is close to a tag.

• I think the revisionMap file where we keep the downloaded revision-to-commit could be removed as this can be done locally by querying the bare repo. The only part the we could be more complicated is knowing which revisions are downloaded in the list command. But this information is not provided in master

Hola Jorge,
let me share a remark with you on the revisionMap file.

There is a difference between using the bare repo vs a self-managed revision map file in how updated revisions can be handled, with regards to user control:

1. With the bare repo, every time the user requests to update a single revision, all of the other downloaded revisions will forcibly have an updated revision -> commit mapping;
2. With a revision map file in the other hand, the user retains full control of which individual revisions -> commits maps to update.

Originally I went for option 2. with this PR, to give users more control. Ultimately 1. is also an option, as long as the behaviour is documented for the users.

Thanks for moving this forward!

@marcodelapierre. Thank you for the remark.
If I am not wrong, the update is caused by checkBareRepo that is fetching all branches/tags everytime that is invoked. I think it could be modifiy to the update to the requested revision by just fetching of the requested revision. So, both implementation could be qeuivalent and we shouldn't maintain the revisions map file.
In both cases, with different users sharing the assets cache, you can not ensure that another user has updated either the repo or the map from your last execution. As commented in other commits, the use of branches can not guarantee reproducibility. happening now.

I have also created an alternative way to create the commit clones. For each commit, it creates a repo in the commit folder where objects are shared with the Bare repo using the objects/info/alternates. It is the same as using the --shared option in the git clone command. With this option, there are no replicated git objects. It just replicates the worktree files and a minimum .git folder (~80K) that stores the references to the alternates, current branch, etc.

I have implemented it in a separate method. I am also wondering if it will be better to implement the AssetsManager changes as a subclass (such as BareAssetsManager) instead of directly modifying it. I am not sure which other components are using the AssetsManager. Implementing the changes as a subclass will not affect other components. @ben, @pditommaso do you think it is a good approach? or do you prefer to keep it as a modification of the AssetsManager class as it is now

@marcodelapierre. Thank you for the remark. If I am not wrong, the update is caused by checkBareRepo that is fetching all branches/tags everytime that is invoked. I think it could be modifiy to the update to the requested revision by just fetching of the requested revision. So, both implementation could be qeuivalent and we shouldn't maintain the revisions map file. In both cases, with different users sharing the assets cache, you can not ensure that another user has updated either the repo or the map from your last execution. As commented in other commits, the use of branches can not guarantee reproducibility. happening now.

Thanks a lot Jorge, I wasn't aware of options to update selected revisions in the bare repo, now it makes total sense to me! :)

Closing in favor of #6620