eif: fix name collision in same folder for exes and melange emits#10220
eif: fix name collision in same folder for exes and melange emits#10220rgrinberg merged 24 commits intoocaml:mainfrom
Conversation
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
src/dune_rules/artifacts_obj.mli
Outdated
| -> libs:(Library.t * _ * Modules.t * Path.Build.t Obj_dir.t) list | ||
| -> exes:(_ * _ * Modules.t * Path.Build.t Obj_dir.t) list | ||
| -> libs:(Library.t * _ * Modules.t * Path.Build.t Obj_dir.t * Toggle.t) list | ||
| -> exes:(_ * _ * Modules.t * Path.Build.t Obj_dir.t * Toggle.t) list |
There was a problem hiding this comment.
It seems like you pass the toggle to artifacts_obj but you don't do anything with it, any reason for that?
There was a problem hiding this comment.
I used it originally, but I guess I can remove it now and just pass the right list of libs, exes and emits directly from the caller, if that's the better approach.
There was a problem hiding this comment.
I'm not sure if it's the better approach, so I'm looking at the code right to find out. I'll list my thought process below. You'll have to repeat in a few different places, so it might be useful.
First, I need to check where are the callers of Artifacts_obj that use a library name. Looks like there's only one caller of:
val lookup_library : t -> Lib_name.t -> Lib_info.local option
That lives in Expander:
(match Artifacts_obj.lookup_library artifacts name with
| None -> does_not_exist ~what:"Library" (Lib_name.to_string name)
| Some lib ->
Mode.Dict.get (Lib_info.archives lib) mode
|> Action_builder.List.map ~f:(fun fn ->
let fn = Path.build fn in
let+ () = Action_builder.path fn in
Value.Path fn))
So now I think about what's going to happen if we pretend that a disabled library does not exist. Since this code is part of the expander, to reproduce this behavior I'd run the following command dune build %{cma:dir/disabled-lib}.
From the perspective of the user, dune telling me this does not exist would be rather undesirable. So we need to improve the situation somehow. To do so, we need Artifacts_obj.lookup_library to propagate the library being disabled.
I make the conclusion that your decision is indeed correct and we need to pass enabled/disabled to Artifacts_obj
Hope that helps.
There was a problem hiding this comment.
Just to make sure I understand, this process would help on the eventual case that in some future Dune sets "fake" rules for all the disabled libraries in the current context right? As we confirmed the other day that there's no need to set rules for disabled libs right now.
I also wonder, in this hypothetic future where there are fake rules for disabled libraries (for better user errors), how would we deal with the case where two libraries are defined in the same folder with the same name, but one defined on each context? I assume we would skip the creation of fake rules in that case?
There was a problem hiding this comment.
Yes, we discussed using fake rules to make the error messages better. And indeed the decision was to forego them for now, as it's quite complex to implement. However, there's other cases where we can improve the error messages without much complexity. Expanding percent forms like we have here is an obvious candidate.
In your hypothetical case we would skip the fake rules. As they would overlap with actual rules.
There was a problem hiding this comment.
I added a test case for this in #10245
Looks like we don't handle disabled libraries correctly
There was a problem hiding this comment.
The change requests didn't specify if this is something that should be fixed in this PR. Is that the case? In any case, the PR doesn't seem to change the current behavior.
There was a problem hiding this comment.
No, there's no need to fix it here.
|
@rgrinberg Sorry, this PR is not yet ready. The CI failures helped me understand there is something borked on the way |
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
ea9def8 to
5189cc4
Compare
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
| @@ -1,4 +1,4 @@ | |||
| enabled_if has a limitation: it attempts building even if enabled_if evaluates to false. | |||
| enabled_if and build_if have similar behavior | |||
There was a problem hiding this comment.
The old behavior was in fact correct and this PR breaks it. The description of this test might be the source of confusion. There's no limitation anywhere, it's just that the different semantics were chosen for enabled_if on tests.
There was a problem hiding this comment.
Just as a side note, if I'm understanding the current behavior properly, it is quite confusing from a user perspective. enabled_if semantics are now different depending on where it's used:
- In
executablestanzas, the field means the executable is not built if it evaluates tofalse - In
teststanzas:- when
allalias is used, the field means the test executable is always built, regardless its evaluated value - but for
runtestalias it means the executable will not be built if it evaluates tofalse(likeexecutablestanzas)
- when
I found this comment that explains the use case for it: #7899 (comment).
It's probably too late, but wonder if it would have been possible to leave the behavior of enabled_if consistent, and allow advanced use cases to use a combination of rule and executable, or add a new field test_if.
There was a problem hiding this comment.
Yes, in hindsight it would be better to do it your way. We could revisit it for 4.0
| Error: Alias "melange" specified on the command line is empty. | ||
| It is not defined in . or any of its descendants. | ||
| [1] |
There was a problem hiding this comment.
These errors might be more meaningful to users than the previous behavior (not building anything because the alias is "empty", but not erroring either).
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
src/dune_rules/ml_sources.ml
Outdated
| @@ -45,6 +45,7 @@ module Modules = struct | |||
| * (Loc.t * Module.Source.t) Module_trie.t | |||
There was a problem hiding this comment.
is it time we make this a record instead? that'd be a whole lot easier to follow what each member of this group_part is.
There was a problem hiding this comment.
I agree. I can do so in a follow-up PR.
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Co-authored-by: Antonio Nuno Monteiro <anmonteiro@gmail.com> Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
| @@ -1,4 +1,4 @@ | |||
| enabled_if has a limitation: it attempts building even if enabled_if evaluates to false. | |||
| enabled_if and build_if have similar behavior | |||
There was a problem hiding this comment.
The old behavior was in fact correct and this PR breaks it. The description of this test might be the source of confusion. There's no limitation anywhere, it's just that the different semantics were chosen for enabled_if on tests.
src/dune_rules/artifacts_obj.mli
Outdated
| -> libs:(Library.t * _ * Modules.t * Path.Build.t Obj_dir.t) list | ||
| -> exes:(_ * _ * Modules.t * Path.Build.t Obj_dir.t) list | ||
| -> libs:(Library.t * _ * Modules.t * Path.Build.t Obj_dir.t * Toggle.t) list | ||
| -> exes:(_ * _ * Modules.t * Path.Build.t Obj_dir.t * Toggle.t) list |
There was a problem hiding this comment.
I added a test case for this in #10245
Looks like we don't handle disabled libraries correctly
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
|
I improved the PR as follows:
Should be good once you add a CHANGES entry |
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
Signed-off-by: Javier Chávarri <javier.chavarri@gmail.com>
|
@rgrinberg Thanks for the improvements! I updated the tests in b22ae6f, and added a changelog entry. Please merge if everything looks fine. |
CHANGES: ### Added - Add link flags to to `ocamlmklib` for ctypes stubs (ocaml/dune#8784, @frejsoya) - Remove some unnecessary limitations in the expansions of percent forms in install stanza. For example, the `%{env:..}` form can be used to select files to be installed. (ocaml/dune#10160, @rgrinberg) - Allow artifact expansion percent forms (`%{cma:..}`, `%{cmo:..}`, etc.) in more contexts. Previously, they would be randomly forbidden in some fields. (ocaml/dune#10169, @rgrinberg) - Allow `%{inline_tests}` in more contexts (ocaml/dune#10191, @rgrinberg) - Remove limitations on percent forms in the `(enabled_if ..)` field of libraries (ocaml/dune#10250, @rgrinberg) - Support dialects in `dune describe pp` (ocaml/dune#10283, @emillon) - Allow defining executables or melange emit stanzas with the same name in the same folder under different contexts. (ocaml/dune#10220, @rgrinberg, @jchavarri) ### Fixed - coq: Delay Coq rule setup checks so OCaml-only packages can build in hybrid Coq/OCaml projects when `coqc` is not present. Thanks to @vzaliva for the test case and report (ocaml/dune#9845, fixes ocaml/dune#9818, @rgrinberg, @ejgallego) - Fix conditional source selection with `select` on `bigarray` in OCaml 5 (ocaml/dune#10011, @moyodiallo) - melange: fix inconsistency in virtual library implementation. Concrete modules within a virtual library can now refer to its virtual modules too (ocaml/dune#10051, fixes ocaml/dune#7104, @anmonteiro) - melange: fix a bug that would cause stale `import` paths to be emitted when moving source files within `(include_subdirs ..)` (ocaml/dune#10286, fixes ocaml/dune#9190, @anmonteiro) - Dune file formatting: output utf8 if input is correctly encoded (ocaml/dune#10113, fixes ocaml/dune#9728, @moyodiallo) - Fix expanding dependencies and locks specified in the cram stanza. Previously, they would be installed in the context of the cram test, rather than the cram stanza itself (ocaml/dune#10165, @rgrinberg) - Fix bug with `dune exec --watch` where the working directory would always be set to the project root rather than the directory where the command was run (ocaml/dune#10262, @gridbugs) - Regression fix: sign executables that are promoted into the source tree (ocaml/dune#10263, fixes ocaml/dune#9272, @emillon) - Fix crash when decoding dune-package for libraries with `(include_subdirs qualified)` (ocaml/dune#10269, fixes ocaml/dune#10264, @emillon) ### Changed - Remove the `--react-to-insignificant-changes` option. (ocaml/dune#10083, @rgrinberg)
CHANGES: ### Added - Add link flags to to `ocamlmklib` for ctypes stubs (ocaml/dune#8784, @frejsoya) - Remove some unnecessary limitations in the expansions of percent forms in install stanza. For example, the `%{env:..}` form can be used to select files to be installed. (ocaml/dune#10160, @rgrinberg) - Allow artifact expansion percent forms (`%{cma:..}`, `%{cmo:..}`, etc.) in more contexts. Previously, they would be randomly forbidden in some fields. (ocaml/dune#10169, @rgrinberg) - Allow `%{inline_tests}` in more contexts (ocaml/dune#10191, @rgrinberg) - Remove limitations on percent forms in the `(enabled_if ..)` field of libraries (ocaml/dune#10250, @rgrinberg) - Support dialects in `dune describe pp` (ocaml/dune#10283, @emillon) - Allow defining executables or melange emit stanzas with the same name in the same folder under different contexts. (ocaml/dune#10220, @rgrinberg, @jchavarri) ### Fixed - coq: Delay Coq rule setup checks so OCaml-only packages can build in hybrid Coq/OCaml projects when `coqc` is not present. Thanks to @vzaliva for the test case and report (ocaml/dune#9845, fixes ocaml/dune#9818, @rgrinberg, @ejgallego) - Fix conditional source selection with `select` on `bigarray` in OCaml 5 (ocaml/dune#10011, @moyodiallo) - melange: fix inconsistency in virtual library implementation. Concrete modules within a virtual library can now refer to its virtual modules too (ocaml/dune#10051, fixes ocaml/dune#7104, @anmonteiro) - melange: fix a bug that would cause stale `import` paths to be emitted when moving source files within `(include_subdirs ..)` (ocaml/dune#10286, fixes ocaml/dune#9190, @anmonteiro) - Dune file formatting: output utf8 if input is correctly encoded (ocaml/dune#10113, fixes ocaml/dune#9728, @moyodiallo) - Fix expanding dependencies and locks specified in the cram stanza. Previously, they would be installed in the context of the cram test, rather than the cram stanza itself (ocaml/dune#10165, @rgrinberg) - Fix bug with `dune exec --watch` where the working directory would always be set to the project root rather than the directory where the command was run (ocaml/dune#10262, @gridbugs) - Regression fix: sign executables that are promoted into the source tree (ocaml/dune#10263, fixes ocaml/dune#9272, @emillon) - Fix crash when decoding dune-package for libraries with `(include_subdirs qualified)` (ocaml/dune#10269, fixes ocaml/dune#10264, @emillon) ### Changed - Remove the `--react-to-insignificant-changes` option. (ocaml/dune#10083, @rgrinberg)
This applies (somehow) the idea mentioned in #10179 (comment).
Part of the fix for #10222.
In order to keep the duplication check active for actual cases where 2 libs are enabled in the same folder, a new field is added to the
group_parttype to keep track of theenablestate of either libs, exes or melange-emits.For cases when a collision is detected when both stanzas are enabled, the previous behavior remains.
If one of the stanzas is disabled, then we filter out the disabled stanzas from the list before proceeding.
I guess we could filter the stanzas upfront, but I believe keeping the disable ones as much as possible might enable use cases where the user is trying to target the artifact of a disabled stanza, so Dune can error out appropriately (see related #10026).Edit: in the end, the stanzas are filtered upfront as there were not many upsides to do it lazily at the moment.