Do depexts status check at opam update time by arozovyk · Pull Request #6489 · ocaml/opam

arozovyk · 2025-04-25T12:07:09Z

Fixes #6461
Fixes #4172

src/state/opamStateTypes.mli

rjbou

This is a first look, code only. I need to test and play with it to complete the review.

master_changes.md

src/state/opamSwitchState.ml

master_changes.md

src/state/opamSysInteract.ml

tests/reftests/run.ml

tests/reftests/depexts.test

src/format/opamSysPkg.ml

src/state/opamSysInteract.ml

rjbou · 2025-07-25T14:39:09Z

src/state/opamSysInteract.ml

                match latest with
                | Some p -> OpamSysPkg.Map.add p ps provides
-                | None -> provides (* Bad pacman output ?? *)
+                | None -> provides (* FIXME: Bad pacman output ?? *)


what is the resaon of the fixme ?

I will have to recheck my tests on Altlinux

src/state/opamRepositoryState.ml

src/state/opamSwitchState.ml

src/client/opamClient.ml

src/state/opamSwitchState.ml

src/state/opamFileTools.mli

src/state/opamSwitchState.ml

src/state/opamRepositoryState.ml

master_changes.md

kit-ty-kate

First "quick" review of the new changes looks good, i didn't see anything egregious.
I would say go ahead with rebase and i'll do another final commit by commit review + local test

src/client/opamClient.ml

src/core/opamSystem.mli

src/core/opamSystem.ml

src/state/opamGlobalState.ml

tests/reftests/legacy-git.test

tests/reftests/no-depexts-sandboxed.unix.test

kit-ty-kate

LGTM modulo some minor things to fix

kit-ty-kate · 2026-02-12T10:50:46Z

tests/reftests/depexts.test

+### : <><> Fetching repository information ><><><><><><><><><><><><><><><><><><><><><>
+### : [default] Initialised
+### : 
+### : <><> Creating initial switch 'comp' (invariant ["comp"]) ><><><><><><><><><><><>
+### : 
+### : <><> Installing new switch packages <><><><><><><><><><><><><><><><><><><><><><>
+### : Switch invariant: ["comp"]
+### : [NOTE] External dependency handling not supported for OS family 'unknown'.
+### :        You can disable this check using 'opam option --global depext=false'
+### : 
+### : <><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+### : -> installed comp.1
+### : Done.


why add comments like this?

good question, @arozovyk ?

src/state/opamSysInteract.ml

kit-ty-kate · 2026-02-12T11:20:29Z

tests/reftests/depext-only.test

+# Return code 2 #
+### opam option depext=false
+Set to 'false' the field depext in global configuration
+### opam install foo --depext-only


I'm not sure i agree this shouldn't do anything. I can see a case for disabling depext by default but selectively installing them when needed.

If it's too complicated to do with the new system though, we should at least display an error message saying to set opam option depext=true and exit <> 0, but doing it automatically for the duration of the command would be best i think.

it's easily doable with

--- a/src/client/opamCommands.ml +++ b/src/client/opamCommands.ml @@ -1875,6 +1875,11 @@ let install cli = else "no-depexts")) else OpamGlobalState.with_ `Lock_none @@ fun gt -> + if depext_only && OpamStateConfig.no_depexts gt then + `Error (true, + Printf.sprintf "Depexts mechanism is disabled, \ + --depext-only can't be used.") + else OpamSwitchState.with_ `Lock_write gt @@ fun st -> let pure_atoms = List.filter_map (function `Atom a -> Some a | _ -> None)

kit-ty-kate · 2026-02-12T11:27:41Z

tests/reftests/depexts.test

+<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+-> retrieved other.dev  (file://${BASEDIR}/other)
+-> installed other.dev
+[NOTE] Requirement for system package other-depext overridden in this switch. Use `opam option 'depext-bypass-="other-depext"'' to revert.


Suggested change

[NOTE] Requirement for system package other-depext overridden in this switch. Use `opam option 'depext-bypass-="other-depext"'' to revert.

[NOTE] Requirement for system package other-depext overridden in this switch. Use `opam option 'depext-bypass-="other-depext"'` to revert.

arozovyk · 2026-02-13T13:33:54Z

tests/reftests/depexts.test

+### : <><> Fetching repository information ><><><><><><><><><><><><><><><><><><><><><>
+### : [default] Initialised
+### : 
+### : <><> Creating initial switch 'comp' (invariant ["comp"]) ><><><><><><><><><><><>
+### : 
+### : <><> Installing new switch packages <><><><><><><><><><><><><><><><><><><><><><>
+### : Switch invariant: ["comp"]
+### : [NOTE] External dependency handling not supported for OS family 'unknown'.
+### :        You can disable this check using 'opam option --global depext=false'
+### : 
+### : <><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+### : -> installed comp.1
+### : Done.


I kept it for reference at some point while enabling/disabling the main behaviour of polling depexts on update/init

It's a developement artifact at this point and should be removed

Suggested change

### : <><> Fetching repository information ><><><><><><><><><><><><><><><><><><><><><>

### : [default] Initialised

### :

### : <><> Creating initial switch 'comp' (invariant ["comp"]) ><><><><><><><><><><><>

### :

### : <><> Installing new switch packages <><><><><><><><><><><><><><><><><><><><><><>

### : Switch invariant: ["comp"]

### : [NOTE] External dependency handling not supported for OS family 'unknown'.

### : You can disable this check using 'opam option --global depext=false'

### :

### : <><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>

### : -> installed comp.1

### : Done.

kit-ty-kate · 2026-02-19T13:07:00Z

tests/reftests/depext-only.test

 - sys-foo
 ### :I:5: No depext
+### opam switch create no-depext --empty
 ### OPAMNODEPEXTS=1 opam install foo --depext-only


Suggested change

### OPAMNODEPEXTS=1 opam install foo --depext-only

### OPAMNODEPEXTS=1 opam install foo --depext-only | sed-cmd echo

…and installed These polling functions are also now no-op if given argument is an empty set. Co-authored-by: Raja Boujbel <raja.boujbel@ocamlpro.com>

It can be used to detect if opam is launched in a sandboxed environment, like when it is launched in inner commands (build, install, remove actions).

…e it Compute and cache depexts availability during repository state loading and integrate this information into switch state management. This enables efficient access to system package availability without repeated system queries. This repository state cache is also updated with 'opam update --depexts'. For pinned packages, during switch state loading (or pin update), systms packages status is retrrieved at the moment and completed to existing one. Co-authored-by: Raja Boujbel <raja.boujbel@ocamlpro.com>

Co-authored-by: Kate <kit-ty-kate@exn.st>

kit-ty-kate · 2026-02-20T14:49:13Z

Igoring the unrelated depexts-alpine CI failure which should be fixed later on our side or in opam-repository

#=== ERROR while compiling conf-clang-format.1 ================================#
"clang-format": command not found.

Thanks a lot!

arozovyk commented Apr 25, 2025

View reviewed changes

src/state/opamStateTypes.mli Outdated Show resolved Hide resolved

kit-ty-kate added this to the 2.5.0~alpha1 milestone Apr 25, 2025

arozovyk force-pushed the depexts_status_on_update_2 branch from a9ed214 to 6be401b Compare April 28, 2025 12:18

arozovyk force-pushed the depexts_status_on_update_2 branch 4 times, most recently from dac6fd8 to 4af5735 Compare June 6, 2025 10:35

kit-ty-kate mentioned this pull request Jun 16, 2025

Void Linux external dependency handling #6548

Open

rjbou reviewed Jun 19, 2025

View reviewed changes

arozovyk force-pushed the depexts_status_on_update_2 branch 10 times, most recently from c715f6a to 48e64e4 Compare June 24, 2025 09:44

arozovyk force-pushed the depexts_status_on_update_2 branch from 48e64e4 to 9d59425 Compare July 2, 2025 17:13

rjbou force-pushed the depexts_status_on_update_2 branch from 9d59425 to 28b4831 Compare July 25, 2025 17:34

rjbou reviewed Jul 28, 2025

View reviewed changes

rjbou mentioned this pull request Jul 30, 2025

Allow detection of installed packages through their virtual names on ALT Linux, RHEL-based and SUSE-based distributions #6431

Open

kit-ty-kate mentioned this pull request Aug 15, 2025

[Haiku] Add missing references to Haiku package manager #6641

Draft

arozovyk force-pushed the depexts_status_on_update_2 branch 3 times, most recently from 32ec8d6 to d311299 Compare August 22, 2025 15:14

rjbou reviewed Aug 22, 2025

View reviewed changes

arozovyk force-pushed the depexts_status_on_update_2 branch 2 times, most recently from 4b63e8f to 61bf34d Compare August 29, 2025 11:41

rjbou added the PRIORITY label Dec 15, 2025

rjbou force-pushed the depexts_status_on_update_2 branch 2 times, most recently from 5e1d836 to 201aae0 Compare February 9, 2026 12:07

rjbou self-assigned this Feb 9, 2026

rjbou force-pushed the depexts_status_on_update_2 branch 2 times, most recently from 37e29e2 to f3d5c95 Compare February 10, 2026 10:25

kit-ty-kate reviewed Feb 10, 2026

View reviewed changes

rjbou force-pushed the depexts_status_on_update_2 branch from 171fa5b to 845b2c6 Compare February 10, 2026 19:37

kit-ty-kate reviewed Feb 12, 2026

View reviewed changes

arozovyk commented Feb 13, 2026

View reviewed changes

rjbou force-pushed the depexts_status_on_update_2 branch 2 times, most recently from de66de8 to ef3c0bc Compare February 18, 2026 18:43

kit-ty-kate reviewed Feb 19, 2026

View reviewed changes

rjbou and others added 14 commits February 19, 2026 18:37

cache: reuse save function

c5f73fc

reftest: add more tests for unknown OS in depexts

fc88983

reftest: add no depexts test

fdda8db

use opam-set-os in tests

9257c44

reftest: add depexts tests that demonstrate system availability polling

905c0c6

Add build_options getter for no_depexts flag

6aaa871

state config: replace no_depexts field by depexts

b4d9a29

OpamSysInteract: add some logging in depext status retrieval

de98c31

Move OpamSysinteract.family type to OpamStateTypes

6737d5f

OpamSysInteract: add disable_depexts_note to harmonise outputs

eddba81

OpamSysInteract: some os_families helpers

c926f52

OpamSysInteract: split system packages status polling into available …

9ba283d

…and installed These polling functions are also now no-op if given argument is an empty set. Co-authored-by: Raja Boujbel <raja.boujbel@ocamlpro.com>

global state: add function to detect if opam root is read-only

b0f55c3

It can be used to detect if opam is launched in a sandboxed environment, like when it is launched in inner commands (build, install, remove actions).

rjbou force-pushed the depexts_status_on_update_2 branch from ef3c0bc to b9b9168 Compare February 19, 2026 21:09

Fix a typo in the note telling users about new a depexts bypass

4822b3e

Co-authored-by: Kate <kit-ty-kate@exn.st>

	[NOTE] Requirement for system package other-depext overridden in this switch. Use `opam option 'depext-bypass-="other-depext"'' to revert.
	[NOTE] Requirement for system package other-depext overridden in this switch. Use `opam option 'depext-bypass-="other-depext"'` to revert.

	### OPAMNODEPEXTS=1 opam install foo --depext-only
	### OPAMNODEPEXTS=1 opam install foo --depext-only \| sed-cmd echo

Conversation

arozovyk commented Apr 25, 2025 • edited by kit-ty-kate Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

rjbou left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arozovyk Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kit-ty-kate left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kit-ty-kate left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kit-ty-kate commented Feb 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

arozovyk commented Apr 25, 2025 •

edited by kit-ty-kate

Loading

rjbou left a comment •

edited

Loading

arozovyk Aug 20, 2025 •

edited

Loading

kit-ty-kate left a comment •

edited

Loading