Skip to content

[Dev] Add "registries" to vcpkg.json, add script to list the packages of the registry.#17124

Merged
Mytherin merged 1 commit intoduckdb:mainfrom
Tishj:add_registries_to_generated_vcpkg_json
Apr 15, 2025
Merged

[Dev] Add "registries" to vcpkg.json, add script to list the packages of the registry.#17124
Mytherin merged 1 commit intoduckdb:mainfrom
Tishj:add_registries_to_generated_vcpkg_json

Conversation

@Tishj
Copy link
Contributor

@Tishj Tishj commented Apr 15, 2025

This PR adds https://github.com/duckdb/vcpkg-duckdb-ports to the generated vcpkg.json file.
This serves as an alternative to the copy-and-pasted ports we had previously.

All of the ports currently in (out of tree) extensions should move to the registry, this avoids needing to copy and paste the port when another extension depends on the same dependency, or even depends on a different extension that has a port.

@Tishj Tishj requested a review from carlopi April 15, 2025 10:46
Copy link
Contributor

@carlopi carlopi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot! This makes sense as-is already, thanks for the step.

There are two (independent) path in which this can move forward:

modernizing current extensions

  1. Copy current ports to vcpkg-duckdb-ports repository (see REAME)
  2. Add (once) a snippet like:
    "registries": [
      {
        "kind": "git",
        "repository": "https://github.com/duckdb/vcpkg-duckdb-ports",
        "baseline": "0f9bf648ba1ee29291890a1ca9a49a80bba017eb",
        "packages": [ 'avro-c', 'vcpkg-cmake' ]
      },
     "builtin-baseline" : "5e5d0e1cd7785623065e77eff011afdeec1a3574"

to vcpkg.json. See reference PR for duckdb-avro at https://github.com/duckdb/duckdb-avro/pull/22/files#diff-dbbb1b147126744a5eee012901d2b9a29cc478be03677f67825b9b2794f5d283

Actual hash AND list of packages should be like current main of duckdb/vcpkg-duckdb-ports and list of all packages.
3. Remove local vcpkg_ports in favor of external ones provided by duckdb/vcpkg-duckdb-ports (this require removing folder from vcpkg_ports and checking dependency is listed in registry.packages
4. Iterate

keep developing extensions

Eventually there will be needs to improve ports.

  1. Move back to local vcpkg_port
  2. Modify however needed
  3. Test
  4. send a PR to duckdb/vcpkg-duckdb-ports (currently missing CI, to be done)
  5. once merge, remove local vcpkg_port and move back to centralized vcpkg-duckdb-ports, new hash
  6. send a PR to duckdb/duckdb to bump the hash

Advantage in centralizing is that most extensions will need the same changes, and that allows also faster diffusion of changes like enabling compilation to Wasm.

@carlopi
Copy link
Contributor

carlopi commented Apr 15, 2025

Obviously here CI on extension building is relevant to double check, but should (in theory) work

@Mytherin Mytherin merged commit f2de8fd into duckdb:main Apr 15, 2025
46 checks passed
@Mytherin
Copy link
Collaborator

Thanks!

@Tishj
Copy link
Contributor Author

Tishj commented Apr 17, 2025

@carlopi I think this might need a follow up
Because I just pasted this same config into duckdb-delta, and got this:

The configuration defined in /Users/thijs/DuckDBLabs/duckdb_delta/vcpkg.json is invalid.

Using registries requires that a baseline is set for the default registry or that the default registry is null.

@carlopi
Copy link
Contributor

carlopi commented Apr 17, 2025

I need to update my comment, when you add registries field you also need to add a builtin-baseline, like:
"builtin-baseline" : "5e5d0e1cd7785623065e77eff011afdeec1a3574"

krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 18, 2025
External File Cache (duckdb/duckdb#16463)
[Dev] Add "registries" to `vcpkg.json`, add script to list the packages of the registry. (duckdb/duckdb#17124)
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 18, 2025
External File Cache (duckdb/duckdb#16463)
[Dev] Add "registries" to `vcpkg.json`, add script to list the packages of the registry. (duckdb/duckdb#17124)
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 19, 2025
External File Cache (duckdb/duckdb#16463)
[Dev] Add "registries" to `vcpkg.json`, add script to list the packages of the registry. (duckdb/duckdb#17124)
@Tishj Tishj deleted the add_registries_to_generated_vcpkg_json branch November 7, 2025 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants