Skip to content

separability and with-constraints: fix for bug #9607#9609

Merged
gasche merged 4 commits intoocaml:trunkfrom
gasche:unboxed-abstract-with-manifest
May 28, 2020
Merged

separability and with-constraints: fix for bug #9607#9609
gasche merged 4 commits intoocaml:trunkfrom
gasche:unboxed-abstract-with-manifest

Conversation

@gasche
Copy link
Copy Markdown
Member

@gasche gasche commented May 27, 2020

This PR fixes #9607, reported by @stedolan (thanks!). The issue is a bit silly: in typedecl, "properties" must be computed in two places (transl_type_decl, but also transl_with_constraint that handles with-constraints), and we didn't know about the second one.

@stedolan would you by chance be willing to review the fix?

@stedolan
Copy link
Copy Markdown
Contributor

The change looks good to me, but it seems like there's an unrelated change to the testsuite that snuck in.

I was wondering whether this bug could occur anywhere else. My intuition is that whenever type_variance is recomputed, type_separability should be computed also (since both have something to do with checking occurrences of type parameters). That makes these lines of typemod.ml seem suspicious - could the bug be there too?

@gasche
Copy link
Copy Markdown
Member Author

gasche commented May 27, 2020

I was already wondering how to get a static check that we are recomputing this in all the right place, but thanks for the other occurrence. I need to think about it more.

@gasche
Copy link
Copy Markdown
Member Author

gasche commented May 27, 2020

@stedolan the code you are pointing out seems to be there to deal with private row types.

Apparently those are handled by a "shadow" type declaration for the row variable, which is fully abstract (and private), followed by a "real" definition for the private type itself, using the shadow row identifier as its row variable. In particular, there is no reason for the private row variable to have a non-default separability (it is purely abstract at this point).

The separability choice there is sound (of course, by default we make the conservative choice of preventing unboxing) but I also have the impression that we could not do better in this case.

I think that the reason why there is a non-trivial variance computed is that you need the variance of the row variable to be compatible with the variance of the rest of the type. (I don't understand the details of the variance computation on pairs of booleans, though.)

@stedolan
Copy link
Copy Markdown
Contributor

Thanks for the explanation!

@stedolan
Copy link
Copy Markdown
Contributor

(the build seems to pass even though a reference file was deleted, so it seems it wasn't needed. leftover from a conversion to expect tests?)

@gasche
Copy link
Copy Markdown
Member Author

gasche commented May 27, 2020

Sorry for not answering earlier. The reference file was dead code (it's mentioned in the commit message that removes it), I suppose from a previous version of the code that did not use expect-style tests (which need no reference file).

Ideally I would still like to make the code more robust so that people adding new kind of variance-like data do not make the same mistake we did, but I think this could and should go in a separate PR. In particular, the current PR will go to 4.11, while a robustification one needs not.

@gasche gasche force-pushed the unboxed-abstract-with-manifest branch from ae65c52 to 9b1bfc3 Compare May 27, 2020 19:32
@gasche
Copy link
Copy Markdown
Member Author

gasche commented May 27, 2020

I tried several approaches to being careful about whether properties are defined or not (based on creating type parameters for them that would be unit in uninitialized type declarations), and they all seemed overkill and too hard to pull through in practice (to compute variance you need to add the partially-defined type declaration to the environment first...). I ended up with a very simple change, which is simply to avoid the dual problem to fragile pattern-matching in a record: { foo with ... }. Given how simple it is, I pushed this as an additional commit to this PR.

@gasche gasche merged commit 4088367 into ocaml:trunk May 28, 2020
gasche added a commit that referenced this pull request May 28, 2020
separability and with-constraints: fix for bug #9607

(cherry picked from commit 4088367)
@gasche
Copy link
Copy Markdown
Member Author

gasche commented May 28, 2020

I merged, and cherry-picked in 4.11 ( 35039ea ).

@hhugo
Copy link
Copy Markdown
Contributor

hhugo commented May 29, 2020

Rebasing on top of that patch, we now get an uncaught exception. I'm not able to provide a repro case yet but I can share the backtrace with you in case it's of any use.
The error was obtain with a ocaml/4.11 compiler that also contains additional/extra commits (janestreet@fac17ac)

[2020-05-29 18:17:45.663151+01:00]       Fatal error: exception Not_found
[2020-05-29 18:17:45.663151+01:00]       Raised at Stdlib__map.Make.find in file "map.ml", line 136, characters 10-25
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_type_full in file "typing/env.ml", line 962, characters 12-49
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_previous in file "typing/ident.ml", line 245, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_previous in file "typing/ident.ml", line 245, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Env.IdTbl.find_same in file "typing/env.ml", line 280, characters 21-30
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_same_module in file "typing/env.ml", line 683, characters 8-30
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_ident_module in file "typing/env.ml", line 854, characters 8-39
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_module_components in file "typing/env.ml", line 861, characters 17-43
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_structure_components in file "typing/env.ml", line 871, characters 23-56
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_type_full in file "typing/env.ml", line 910, characters 15-46
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_type in file "typing/env.ml" (inlined), line 999, characters 2-24
[2020-05-29 18:17:45.663151+01:00]       Called from Typedecl_unboxed.get_unboxed_type_representation in file "typing/typedecl_unboxed.ml", line 31, characters 16-35
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Stdlib__map.Make.find in file "map.ml", line 136, characters 10-25
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_type_full in file "typing/env.ml", line 962, characters 12-49
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_previous in file "typing/ident.ml", line 245, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_previous in file "typing/ident.ml", line 245, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Env.IdTbl.find_same in file "typing/env.ml", line 280, characters 21-30
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_same_module in file "typing/env.ml", line 683, characters 8-30
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_ident_module in file "typing/env.ml", line 854, characters 8-39
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_module_components in file "typing/env.ml", line 861, characters 17-43
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_structure_components in file "typing/env.ml", line 871, characters 23-56
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_type_full in file "typing/env.ml", line 910, characters 15-46
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_type in file "typing/env.ml" (inlined), line 999, characters 2-24
[2020-05-29 18:17:45.663151+01:00]       Called from Ctype.immediacy in file "typing/ctype.ml", line 4827, characters 22-41
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Stdlib__map.Make.find in file "map.ml", line 136, characters 10-25
[2020-05-29 18:17:45.663151+01:00]       Called from Typedecl_separability.Hyps.find in file "typing/typedecl_separability.ml", line 420, characters 24-43
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Stdlib__map.Make.find in file "map.ml", line 136, characters 10-25
[2020-05-29 18:17:45.663151+01:00]       Called from Typedecl_separability.Hyps.find in file "typing/typedecl_separability.ml", line 420, characters 24-43
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Stdlib__map.Make.find in file "map.ml", line 136, characters 10-25
[2020-05-29 18:17:45.663151+01:00]       Called from Typedecl_separability.Hyps.find in file "typing/typedecl_separability.ml", line 420, characters 24-43
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Stdlib__map.Make.find in file "map.ml", line 136, characters 10-25
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_type_full in file "typing/env.ml", line 962, characters 12-49
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_previous in file "typing/ident.ml", line 245, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_previous in file "typing/ident.ml", line 245, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Ident.find_same in file "typing/ident.ml", line 251, characters 6-21
[2020-05-29 18:17:45.663151+01:00]       Called from Env.IdTbl.find_same in file "typing/env.ml", line 275, characters 10-40
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Env.IdTbl.find_same in file "typing/env.ml", line 280, characters 21-30
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_same_module in file "typing/env.ml", line 683, characters 8-30
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_ident_module in file "typing/env.ml", line 854, characters 8-39
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_module_components in file "typing/env.ml", line 861, characters 17-43
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_structure_components in file "typing/env.ml", line 871, characters 23-56
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_type_full in file "typing/env.ml", line 910, characters 15-46
[2020-05-29 18:17:45.663151+01:00]       Called from Env.find_type in file "typing/env.ml" (inlined), line 999, characters 2-24
[2020-05-29 18:17:45.663151+01:00]       Called from Typedecl_separability.check_type.check_type in file "typing/typedecl_separability.ml", line 506, characters 19-43
[2020-05-29 18:17:45.663151+01:00]       Called from Typedecl_separability.check_def in file "typing/typedecl_separability.ml", line 681, characters 6-50
[2020-05-29 18:17:45.663151+01:00]       Called from Typedecl_separability.compute_decl in file "typing/typedecl_separability.ml" (inlined), line 687, characters 7-25
[2020-05-29 18:17:45.663151+01:00]       Called from Typedecl.transl_with_constraint in file "typing/typedecl.ml", line 1452, characters 8-51
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint.merge in file "typing/typemod.ml", line 517, characters 10-72
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint.merge in file "typing/typemod.ml", line 577, characters 28-77
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint.merge in file "typing/typemod.ml", line 577, characters 28-77
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint.merge in file "typing/typemod.ml", line 577, characters 28-77
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint.merge in file "typing/typemod.ml", line 577, characters 28-77
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint.merge in file "typing/typemod.ml", line 577, characters 28-77
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint.merge in file "typing/typemod.ml", line 577, characters 28-77
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint.merge in file "typing/typemod.ml", line 577, characters 28-77
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint.merge in file "typing/typemod.ml", line 577, characters 28-77
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.merge_constraint in file "typing/typemod.ml", line 583, characters 22-53
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.transl_modtype_aux.(fun) in file "typing/typemod.ml", line 1183, characters 14-72
[2020-05-29 18:17:45.663151+01:00]       Called from Stdlib__list.fold_left in file "list.ml", line 121, characters 24-34
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.transl_modtype_aux in file "typing/typemod.ml", line 1180, characters 8-261
[2020-05-29 18:17:45.663151+01:00]       Called from Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 237, characters 14-18
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 242, characters 4-13
[2020-05-29 18:17:45.663151+01:00]       Called from Stdlib__option.map in file "option.ml", line 24, characters 57-62
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.transl_modtype_decl_aux in file "typing/typemod.ml", line 1504, characters 4-69
[2020-05-29 18:17:45.663151+01:00]       Called from Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 237, characters 14-18
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 242, characters 4-13
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.transl_signature.transl_sig in file "typing/typemod.ml", line 1387, characters 34-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.transl_signature.transl_sig in file "typing/typemod.ml", line 1415, characters 41-63
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.transl_signature.(fun) in file "typing/typemod.ml", line 1487, characters 36-77
[2020-05-29 18:17:45.663151+01:00]       Called from Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 237, characters 14-18
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 242, characters 4-13
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.transl_modtype_aux in file "typing/typemod.ml", line 1142, characters 15-39
[2020-05-29 18:17:45.663151+01:00]       Called from Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 237, characters 14-18
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 242, characters 4-13
[2020-05-29 18:17:45.663151+01:00]       Called from Stdlib__option.map in file "option.ml", line 24, characters 57-62
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.transl_modtype_decl_aux in file "typing/typemod.ml", line 1504, characters 4-69
[2020-05-29 18:17:45.663151+01:00]       Called from Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 237, characters 14-18
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 242, characters 4-13
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_str_item in file "typing/typemod.ml", line 2336, characters 30-64
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.type_struct in file "typing/typemod.ml", line 2440, characters 44-68
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure.run in file "typing/typemod.ml", line 2445, characters 33-53
[2020-05-29 18:17:45.663151+01:00]       Called from Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 237, characters 14-18
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 242, characters 4-13
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_structure in file "typing/typemod.ml" (inlined), line 2462, characters 21-46
[2020-05-29 18:17:45.663151+01:00]       Called from Typemod.type_implementation.(fun) in file "typing/typemod.ml", line 2637, characters 8-68
[2020-05-29 18:17:45.663151+01:00]       Called from Misc.try_finally in file "utils/misc.ml", line 31, characters 8-15
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Misc.try_finally in file "utils/misc.ml", line 45, characters 10-56
[2020-05-29 18:17:45.663151+01:00]       Called from Misc.try_finally in file "utils/misc.ml", line 31, characters 8-15
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Misc.try_finally in file "utils/misc.ml", line 45, characters 10-56
[2020-05-29 18:17:45.663151+01:00]       Called from Compile_common.typecheck_impl in file "driver/compile_common.ml", line 104, characters 2-131
[2020-05-29 18:17:45.663151+01:00]       Called from Compile_common.implementation.(fun) in file "driver/compile_common.ml", line 120, characters 18-44
[2020-05-29 18:17:45.663151+01:00]       Called from Misc.try_finally in file "utils/misc.ml", line 31, characters 8-15
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Misc.try_finally in file "utils/misc.ml", line 45, characters 10-56
[2020-05-29 18:17:45.663151+01:00]       Called from Misc.try_finally in file "utils/misc.ml", line 31, characters 8-15
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Misc.try_finally in file "utils/misc.ml", line 45, characters 10-56
[2020-05-29 18:17:45.663151+01:00]       Called from Misc.try_finally in file "utils/misc.ml", line 31, characters 8-15
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Misc.try_finally in file "utils/misc.ml", line 45, characters 10-56
[2020-05-29 18:17:45.663151+01:00]       Called from Compenv.process_action.impl in file "driver/compenv.ml", line 647, characters 4-57
[2020-05-29 18:17:45.663151+01:00]       Called from Stdlib__list.iter in file "list.ml", line 110, characters 12-15
[2020-05-29 18:17:45.663151+01:00]       Called from Compenv.process_deferred_actions in file "driver/compenv.ml", line 729, characters 2-61
[2020-05-29 18:17:45.663151+01:00]       Called from Optmain.main in file "driver/optmain.ml", line 55, characters 6-163
[2020-05-29 18:17:45.663151+01:00]       Re-raised at Location.report_exception.loop in file "parsing/location.ml", line 926, characters 14-25
[2020-05-29 18:17:45.663151+01:00]       Called from Optmain.main in file "driver/optmain.ml", line 133, characters 6-37
[2020-05-29 18:17:45.663151+01:00]       Called from Optmain in file "driver/optmain.ml", line 137, characters 2-9

@gasche
Copy link
Copy Markdown
Member Author

gasche commented May 30, 2020

It took me a bit of time to figure it out (but most of the time was spent trying to write a correct emacs-lisp regexp to parse the new backtrace format to easily jump around the trace), but I suspect that the reason is that the source uses a type construction from a compilation unit whose .cmi is missing from the load path.

The trace is a bit hard to read due to the fact that your compiler variant uses reraise by default (but then this is an interesting choice worth discussing upstream: when we debug, maybe it is better to have too much information than not enough information?). What we see is not a single trace, but several traces concatenated for the Not_found exception. The last one was uncaught, and I believe it does come from typedecl_separability, but the previous one were caught and handled (but they remained in the backtrace state). In this case the stale traces were helpful to help me understand the issue, by comparing with another site that raises Not_found.

The place where the exception is raised looks like this in Typedecl_separability, which handles [@@unboxed] attributes:

    | (Tconstr(path,tys,_), m      ) ->
        let msig = (Env.find_type path env).type_separability in
        ...

but earlier in the stale part of trace we can find a place in Ctype.immediacy (which handles [@@immediate]) where Not_found was raised and caught:

  | Tconstr(p, _args, _abbrev) ->
    begin try
      let type_decl = Env.find_type p env in
      type_decl.type_immediate
    with Not_found -> Type_immediacy.Unknown
    (* This can happen due to e.g. missing -I options,
       causing some .cmi files to be unavailable.
       Maybe we should emit a warning. *)
    end

Here is my current hypothesis for what is happening: you have an [@@unboxed] [@@immediate] declaration whose type constructor belongs to a missing compilation unit.

I would have preferred if the "immediacy" function had emitted a warning indeed, because this would have been helpful to locate the problem. I will have to add the same workaround/hack in typedecl_separability.ml.

@gasche
Copy link
Copy Markdown
Member Author

gasche commented May 30, 2020

I don't know how to write a testcase for this potential failure. My attempt, taking inspiration from testsuite/typing-missing-cmi-3, failed.

@gasche
Copy link
Copy Markdown
Member Author

gasche commented May 30, 2020

@hhugo could you try redoing the build with the following workaround ?

gasche@2a142de

@hhugo
Copy link
Copy Markdown
Contributor

hhugo commented Jun 1, 2020

gasche@2a142de does fix the uncaught exception but it seems I still hit an occurrence of #9607. I'll try to get you a repro case.

@gasche
Copy link
Copy Markdown
Member Author

gasche commented Jun 1, 2020

Thanks!

@hhugo
Copy link
Copy Markdown
Contributor

hhugo commented Jun 1, 2020

The following use to compile and no longer does. I'm not sure how much it's related to #9607

module Private_variant = struct

  type 'a t = A : 'e -> 'e t

end

module type Public = sig
  module Event : sig
    type 'e t = 'e Private_variant.t
  end
  type t = T : 'e Event.t -> t [@@unboxed]
end

module type Component = sig

  include Public

  module type Public = Public
    with type 'e Event.t = 'e Event.t
     and type t = t
end

Here is the error I get with your latest patch

File "/home/hheuzard/a.ml", line 20, characters 9-19:
20 |      and type t = t
              ^^^^^^^^^^
Error: This type cannot be unboxed because
       it might contain both float and non-float values,
       depending on the instantiation of an unnamed existential variable.
       You should annotate it with [@@ocaml.boxed].

@gasche
Copy link
Copy Markdown
Member Author

gasche commented Jun 1, 2020

Thanks! This is great, because it also gives a repro case for the Not_found error we had earlier. The comment from Type_immediacy suggests that the Not_found error was due to missing cmi phenonema, but I had been unable to reproduce it by playing with removing cmi. I see now that your examples generate a Not_found exception (without my proposed patch), so this suggests that the Not_found issue (in our code and in Type_immediacy) is in fact not related to missing .cmi (or not only?), and the catch-all was hiding a different bug (wrong environment passed somewhere) in these module constructs.

@gasche
Copy link
Copy Markdown
Member Author

gasche commented Jun 1, 2020

@hhugo could you open a new issue with this repro case? I believe that there is an interesting bug lying somewhere around it (possibly also affecting immediacy and variance), and it would be nice to have a new PR number to refer to it and potential fixes.

@gasche
Copy link
Copy Markdown
Member Author

gasche commented Jun 1, 2020

A slightly simpler repro case:

module type Sig = sig
  type 'a ind = 'a * int
  type t = T : 'e ind -> t [@@unboxed]
end

module type Problem = sig
  include Sig
  module type ReboundSig = Sig
    with type 'a ind = 'a ind
     and type t = t
end

@gasche
Copy link
Copy Markdown
Member Author

gasche commented Jun 1, 2020

I opened a draft PR with a preliminary fix in #9623.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unboxability check breaks with module constraints

3 participants