Skip to content

Fix reproducibility for -no-alias-deps#9991

Merged
lpw25 merged 2 commits intoocaml:trunkfrom
lpw25:reproducible-no-alias-deps
Nov 22, 2020
Merged

Fix reproducibility for -no-alias-deps#9991
lpw25 merged 2 commits intoocaml:trunkfrom
lpw25:reproducible-no-alias-deps

Conversation

@lpw25
Copy link
Copy Markdown
Contributor

@lpw25 lpw25 commented Oct 26, 2020

#9345 makes environment summaries reproducible by only adding Env_persistent in cases where the entry materially changes the environment. This ensures that cmt files do not depend on which unrelated cmi files are on the filesystem.

However, such unrelated cmis still affect sharing of strings in initial environment itself. Normally this sharing is not observable because if you try to look up a module for which there is no cmi then you will error. However, with -no-alias-deps we allow look ups of module aliases to succeed even if there is no cmi, which reveals the difference in sharing and leads to reproducibility issues.

This PR changes the behaviour in -no-alias-deps mode to only add persistent modules to the initial environment if doing so would materially change the environment.

It also modifies the reproducibility test in the test suite to trigger this case.

@lpw25 lpw25 marked this pull request as draft October 26, 2020 13:20
Copy link
Copy Markdown
Member

@gasche gasche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which "strings" are we talking about here? (I think you mean the strings in the type-checker that are part of the compilation-unit metadata; and in particular not string constants in the module itself, right?)

Why is there no new testsuite example that checks something on strings somehow?

@lpw25 lpw25 force-pushed the reproducible-no-alias-deps branch from bbd3956 to 783150a Compare October 26, 2020 14:34
Copy link
Copy Markdown
Member

@gasche gasche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't really understand the explanation for the PR, but I think that the implementation makes sense.

@lpw25
Copy link
Copy Markdown
Contributor Author

lpw25 commented Oct 26, 2020

The "strings" are the names of persistent modules within identifiers. If the cmi file is present then the string in the identifier in the environment comes from the read of the file system. If the cmi file is not present then the string in the identifier comes from the source code at each point where the file is referenced, and is shared with the other occurrences of that string in the cmt file.

@lpw25 lpw25 force-pushed the reproducible-no-alias-deps branch from 783150a to d607057 Compare October 26, 2020 15:00
Copy link
Copy Markdown
Member

@gasche gasche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR now looks good to me. (But please consider addressing @Octachron question(s) before merging.)

@gasche gasche added the bug label Nov 3, 2020
@lpw25 lpw25 force-pushed the reproducible-no-alias-deps branch from d607057 to 0166a70 Compare November 22, 2020 13:47
@lpw25 lpw25 marked this pull request as ready for review November 22, 2020 13:47
@lpw25
Copy link
Copy Markdown
Contributor Author

lpw25 commented Nov 22, 2020

Rebased, added a comment and added a Changes entry.

@lpw25 lpw25 merged commit d0ddf25 into ocaml:trunk Nov 22, 2020
@Octachron
Copy link
Copy Markdown
Member

This fix seems specialized enough to be backported to 4.12. Any objections?

@gasche
Copy link
Copy Markdown
Member

gasche commented Nov 23, 2020

None.

Octachron pushed a commit that referenced this pull request Nov 24, 2020
Fix reproducibility for `-no-alias-deps`

(cherry picked from commit d0ddf25)
@Octachron
Copy link
Copy Markdown
Member

Cherry-picked to 4.12 as 1d2363f .

Octachron pushed a commit to Octachron/ocaml that referenced this pull request Nov 25, 2020
Fix reproducibility for `-no-alias-deps`

(cherry picked from commit d0ddf25)
@Octachron
Copy link
Copy Markdown
Member

For records: an unforeseen consequence of this PR is that more cmi files compiled with and without -no-alias-deps might be mismatched: there is a opam package (sundialsml) that triggers this problem by compiling a ml-only module

external f: M.x -> M.x

with -no-alias-deps in bytecode mode and without it in native mode.
However, relying on cmi equality in presence of different compiler flags seems far too brittle, patching the offending package seems to be the better course of action here.

@tbrk
Copy link
Copy Markdown
Contributor

tbrk commented Mar 14, 2021

We just came across this problem when building Sundials/ML with OCaml 4.12.0 as @Octachron predicted! The lack of -no-alias-deps for the native mode build was simply an oversight that is now corrected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants