Skip to content

Report exits from the toplevel correctly#382

Merged
Leonidas-from-XIV merged 8 commits intorealworldocaml:mainfrom
talex5:toplevel-exit
Jul 21, 2022
Merged

Report exits from the toplevel correctly#382
Leonidas-from-XIV merged 8 commits intorealworldocaml:mainfrom
talex5:toplevel-exit

Conversation

@talex5
Copy link
Copy Markdown
Contributor

@talex5 talex5 commented Jul 8, 2022

Previously, you just got:

ocaml-mdx-test: internal error, uncaught exception:
                Compenv.Exit_with_status(125)

Previously, you just got:

    ocaml-mdx-test: internal error, uncaught exception:
                    Compenv.Exit_with_status(125)
@Leonidas-from-XIV
Copy link
Copy Markdown
Collaborator

Looks like there is a multi-version incompatibility:

  • 4.08 to 4.11 don't have Compenv.Exit_with_status
  • 4.12 to 4.13 don't throw errors on invalid #use
  • 4.14 works

It's quite hard to build a compatibility wrapper in Compat_top to work around this if there isn't a consistent way to get the error between different versions, especially since before < 4.14 these don't seem to trigger errors.

I looked a bit around in the source code of 4.12 vs 4.14 and it seems like execute_phrase has seen some considerable rework between the releases which has probably lead to better error reporting.

@Leonidas-from-XIV
Copy link
Copy Markdown
Collaborator

I played around a bit in #384 to get it to compile on older versions, which is not too difficult. Two issues remain:

  1. The output disappears. This seems to be because in <4.14 #use is hardcoded to write to std_out whereas in 4.14 it will write to stdout or stderr. I think it should be possible to patch #use to use stderr on older versions too, if we just redefine the relevant directives.
  2. No Compenv.Exit_with_status exception is thrown. I have yet to figure out how this happens in 4.14, because even despite the directive failing and printing to the void, older versions happily continue on, even returning true in the evaluation because directives always evaluate to true.

They have to be redirected because by default they write to stdout, but
we capture stderr (and in 4.14 they write to stderr). Unfortunately,
just overwriting them in the directives Hashtbl does not work, since
the ordering of additions is somewhat undefined so they might be
overwritten. A safer way is to create new directives and rewrite those
to be evaluated to them.

Also, `"use"` is a builtin directive, not from findlib so needs to be
excluded to get the error message printed.
`use_output` was added in 4.11, whereas the `use_trace` was removed in
4.13.
The stdlib pre-4.14 defines them to be `ignore`d but they can be made to
throw the error like in 4.14. At least some of them, for which the
underlying functions are exposed in the interface.
OCaml 4.13 introduces a few deprecations that we need to work around.
@Leonidas-from-XIV
Copy link
Copy Markdown
Collaborator

Okay, I looked into this and I think I resolved all the issues here:

  1. Redefined a lot of directives (those exposed in Topdirs) to write to stderr instead of stdout, so they get captured. 4.14 does that conditionally, but here we expect it to be always to be on stderr. There's some subtlety to this because depending on the supported version this looks different. In particular, #load can only be overridden in 4.13+ because older versions don't expose load_file which does not recurse. We could try to copy-paste the definition but I opted to avoid copying too much code and keep the complexity (somewhat) low at the cost of less than 100% identical behavior.
  2. The redefined directives no longer use ignore to suppress the error but throw Exit_with_status, similar to how it is done on 4.14, but with our own exception so it works on older OCaml versions as well.

The directives are all implemented as custom mdx_<original-name> directives, because when I tried to override the existing ones there is a race condition between the original and the patched ones. So instead the code rewrites e.g. #use into #mdx_use on OCaml <4.14 which avoids the headache altogether.

This was a fun challenge, teaching me a thing or two how the directives in the toplevel work.

@talex5
Copy link
Copy Markdown
Contributor Author

talex5 commented Jul 14, 2022

Thanks - I hadn't realised how much work it would be to support older versions!

Copy link
Copy Markdown
Contributor

@NathanReb NathanReb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks to both of you for fixing this!

It looks good to me. My only concern is that the Compat_top module is a tiny bit hard to follow at the moment but I'm not sure how to exactly improve it so don't consider this a blocker.

@Leonidas-from-XIV
Copy link
Copy Markdown
Collaborator

I could move stuff out of Compat_top but besides being "somewhere else" I feel there is not particularly much that can be done, given this is all icky reimplementing of internal libs, trying to duplicate the least amount of code.

Thanks for finding the issues with the constraints, looks like I copypasted exactly the one line that had a problem.

@Leonidas-from-XIV Leonidas-from-XIV merged commit cdcc344 into realworldocaml:main Jul 21, 2022
Leonidas-from-XIV added a commit to Leonidas-from-XIV/opam-repository that referenced this pull request Jan 6, 2023
CHANGES:

#### Added

- Report all parsing errors in Markdown files (realworldocaml/mdx#389, @NathanReb)

#### Changed

- Preserve indentation in multiline OCaml blocks in .mli files (realworldocaml/mdx#395, @panglesd)

#### Fixed

- Fixed compatibility with Cmdliner 1.1.0 (realworldocaml/mdx#371, @Leonidas-from-XIV)
- Report errors and exit codes of toplevel directives (realworldocaml/mdx#382, @talex5,
  @Leonidas-from-XIV)
- Fix block locations in error reporting (realworldocaml/mdx#389, @NathanReb)
- Include the content of the line that features the `part-end` MDX directive in
  the output, before that line would've been dropped (realworldocaml/mdx#374, realworldocaml/mdx#387,
  @Leonidas-from-XIV)
- Handle EINTR signal on waitpid call by restarting the syscall. (realworldocaml/mdx#409, @tmcgilchrist)
- Fix parsing of multiline toplevel phrases in .mli files (realworldocaml/mdx#394, realworldocaml/mdx#397,
  @Leonidas-from-XIV)

#### Removed

- Removed warning about missing semicolons added in MDX 1.11.0 and the
  automatic insertion of semicolons in the corrected files introduced in MDX
  2.0.0. (realworldocaml/mdx#398, @Leonidas-from-XIV)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants