Skip to content

Idea/Proposal: Explicit Support for Vendoring #1016

@rgrinberg

Description

@rgrinberg

Problem

dune's modularity allows for seamless vendoring. This is a good starting point, but it does reveal some weaknesses of the current naive approach. This proposal tries to motivate the benefits for creating a mechanism to mark directories as "vendored" to allow for dune to treat this directories in some special ways. As an initial proposal, I suggest a (vendored) stanza that will mark the directory its defined (recursively) as vendored.

The end goal of this proposal to make it transparent to switch a dependency from an external to vendored mode and to make it trivial to update vendored deps. That is without modifying any build files or sources.

Automatic exclusion from coverage and profiling tools

This is for the upcoming bisect support.

Enable the env stanza for workspaces that include more than one scope

The reason why (env ..) doesn't work for configuring all opam packages in a scope is that it would be incorrect to apply such customizations to vendored packages. If we know which directories are vendored, we'll be able to lift this general restriction on env and only apply on the vendored dirs.

Automatic adjustment of the @test alias

The tests for the vendored dependencies can automatically be disabled. This is pretty obvious.

Automatic adjustment of the @install alias

Currently, naive vendoring will pollute the @install alias with artifacts defined in vendored libs/exes. These artifacts should be as hidden as possible and hence automatically treated as private. This is already something that users currently do by stripping the public_name fields from their vendored packages. If dune knows which packages are vendored it would do this adjustment automatically (not include the vendored artifacts in @install) and would still allow the public_names of vendored artifacts to be used.

Package private names for vendored dependencies

Today, if one wants to vendor a library then the library should be renamed so that the package doesn't conflict with packages that use the non vendored of the package. If dune understands which libraries are vendored, then it will be able to this renaming automatically. I don't want to spend time describing something elaborate here, so here's a simple description for how it would work for (wrapped false) vendored libraries. It should be easy enough to extend this to wrapped libraries as well.

  • The (wrapped false) library will now be (wrapped true)
  • Some random name would be generate for the lib name E.g. vendored_lib<digest>.
  • Users of this vendored library in the project will automatically get the -open vendored_lib<digest> flag which will make it possible to use the unmangled names.

The main issue with this approach is that it doesn't really work for vendored libs with c_stubs. We can either just disable this kind of name mangling for libs with c stubs or just warn people about this issue.

I'm cc'ing some prominent users of vendored dependencies for feedback: @avsm @samoht and @jordwalke

Metadata

Metadata

Assignees

No one assigned

    Labels

    proposalRFC's that are awaiting discussion to be accepted or rejected

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions