Skip to content

Add colors to type clash error message#10321

Closed
EduardoRFS wants to merge 3 commits intoocaml:trunkfrom
EduardoRFS:colorful-error-messages
Closed

Add colors to type clash error message#10321
EduardoRFS wants to merge 3 commits intoocaml:trunkfrom
EduardoRFS:colorful-error-messages

Conversation

@EduardoRFS
Copy link
Copy Markdown
Contributor

Problem

Currently OCaml messages lacks visual clues on type clashes which forces the user to actually read the error message to understand what was expected and what is received.

Solution

By making the received type red and the expected type green, it highlights the important parts of the message and what should be addressed. Received is red because it is the wrong type, and expected is green because it is the right type.

Note that the arrow pointing out to the location is already red to give the same kind of visual clue.

Example

images with error message showing received type with color red and expected type with color green

@EduardoRFS
Copy link
Copy Markdown
Contributor Author

@Octachron hopefully this is not controversial

@gasche
Copy link
Copy Markdown
Member

gasche commented Apr 5, 2021

Not controversial? A PR about colors? Of course this will be debated forever :-)

Received is red because it is the wrong type, and expected is green because it is the right type.

I don't agree with the reasoning in general. The expected type may be completely wrong, due to a programmer mistake somewhere else in the code.

# 1.5 + 2.5;;
  ^^^
Error: This expression has type float but an expression was expected of type
         int

Before doing any further bikeshedding, two simple questions:

  • Could you post a "before" screenshot for the same example? what does the printing currently look like?
  • Have you tried printing the two types in bold / emphasized, to make them stand out visually, without having them use different colors?

@EduardoRFS
Copy link
Copy Markdown
Contributor Author

@gasche

I don't agree with the reasoning in general.

It's just reflecting the reasoning in the error message, in the sense of "if you provide the compiler with green, the code builds".

Red and green still looks easier to me to track, especially the green one. But we can try with any color, just give me the combination that you want, and I will do it.

Before

current error message with no additional colors

After

new error messages with the changes of the PR

Bold

error messages with the suggestion made by @gasche the colors are bold

@yawaramin
Copy link
Copy Markdown
Contributor

Should this be applied to interface/implementation mismatch errors? Relevant: #7574

@EduardoRFS
Copy link
Copy Markdown
Contributor Author

@yawaramin I think so, both Includemod and Includeclass also have clashes, and there is a method clash on Printtyp.

But that's another discussion, just trying to see if we can agree on the base case.

@Drup
Copy link
Copy Markdown
Contributor

Drup commented Apr 5, 2021

The use of colors and how they are used seem consistent with #9331 so I would tend to be in favor.

In any case, just bold is already quite better.

@dbuenzli
Copy link
Copy Markdown
Contributor

dbuenzli commented Apr 5, 2021

Yet another scheme:

  1. Print the type of the expression in a color that matches the color in which the code is highlighted (that seems to be red).
  2. Print the expected type in a neutral bold.

@EduardoRFS
Copy link
Copy Markdown
Contributor Author

@dbuenzli so essentially green to bold?

I'm still on the team of "green is the goal", which may be misleading sometimes, but that's the nature of an error message.

image showing error message with expected type bold white and received type red

@johnwhitington
Copy link
Copy Markdown
Contributor

Red-Green colourblindness is the most common type.

@EduardoRFS
Copy link
Copy Markdown
Contributor Author

EduardoRFS commented Apr 5, 2021

@johnwhitington it uses the colors from the terminal, and I expect anyone who is colorblind to change their respective colors in the terminal. So I think this is not affected by this problem.

Also if I understood correctly it would look grey to someone who is colorblind, so they can hopefully still be able to read it and notice the problem.

@EduardoRFS
Copy link
Copy Markdown
Contributor Author

EduardoRFS commented Apr 5, 2021

There is another case to discuss, which is when the message has the unfolded type, saying "type X is not compatible with type Y"

Using the same color for both is weird.

example of both expected types and both received types in the same respective color, weird

My suggestion is, when this happens the main type diff uses bold, and the unfolded part is actually red and green.

example of complete expected and received type bold and the diff in the colors red and green

@hcarty
Copy link
Copy Markdown
Member

hcarty commented Apr 5, 2021

I like the use of matching colors in the last example. If one is not red/green color blind it's immediately visible that float comes from the first type and int from the second.

Having all of the types bold helps in any case as @dbuenzli mentioned.

@Octachron
Copy link
Copy Markdown
Member

One possibility would be to choose a pair of color less associated with error/success to focus on the existence of two sides: For instance red/blue or orange/blue seems color-blind friendly?

@yakobowski
Copy link
Copy Markdown
Contributor

IMO, having the inferred type printed with the same colour as Error· has value (for consistency). Also, for multi-line messages, highlighting everything consistently seems better too.

@ulugbekna
Copy link
Copy Markdown
Contributor

ulugbekna commented Apr 8, 2021

I like the PR, but is it solving the right problem? I think that we need those colors and we debate about those colors because a type clash error message is verbose.

We could solve the problem in a more fundamental way by making the error message more clear. I think that

This expression has type int but an expression was expected of type string.

could be replaced by

Type mismatch. 
  Found: int
  Required: string

The type clash in the latter case is even more clear than the former with colors imo.

One might say that the error message is less understandable, but looking at type error messages of Rust (the most loved lang for several years) and Scala (popular FP lang) we can see that they have something very close to the latter.

Scala:

image

Rust:

image

OCaml :-)

image

Unfortunately, examples are not from consistent sources (repl vs compiler vs ide).

Apologies for derailing the conversation from the PR.

@gasche
Copy link
Copy Markdown
Member

gasche commented Apr 8, 2021

(I posted a comment in #9328 which I meant for here, apologies.)

What is the meaning of the color schemes used in #9331? If I understand correctly, green is used when an argument matches the expected type (of its corresponding formal parameter), magenta is used for both the argument and the formal parameter when there is a mismatch, and red is used when an argument (or formal parameter) has no corresponding formal parameter (or argument).

In the present PR, we don't have a case where a type matches its expectation (where they would both be green); if we reused the colors of the example example, I guess the two types would be displayed in magenta: we have exactly one expectation and one provided type, and they don't match.

Note: I like @ulugbekna's proposal (I would use "expected" rather than "required").

@gasche
Copy link
Copy Markdown
Member

gasche commented Apr 8, 2021

For me, none of the specific color proposals in the PR are clearly better than just using bold, which seems to make a consensus as a first improvement. I think there would be an argument for proposing and merging that first, and refining with other ideas later, including @ulugbekna's proposal for a much shorter message.

@EduardoRFS
Copy link
Copy Markdown
Contributor Author

EduardoRFS commented Apr 8, 2021

Is it solving the right problem?

@ulugbekna No, it is solving a problem. I'm proposing a small improvement and I'm conscious that there is bigger possible improvements, but I don't think we're going to find a consensus on any format change soon and I'm not willing to push for it.

which seems to make a consensus as a first improvement

@gasche Agreed, @Octachron and @dbuenzli is it okay for you the bold one, as shown in the comment above? If so I will do the proper implement and we can start to discuss on it so that hopefully this can happens for 4.13.

@Octachron
Copy link
Copy Markdown
Member

I agree that having a bold font for types (and inline code in general) is already an improvement indeed.

@giltho
Copy link
Copy Markdown
Contributor

giltho commented Apr 13, 2021

Hi,
I'm interested in understanding more about the arguments against adding colour. Of course, colour blindness has to be taken into account but as @EduardoRFS said, terminal colours can be changed. Moreover, why not make it bold AND red/green, or add an environment variable that could be set by colour blind people like OCAML_NO_COLOR which would remove the colours. Basically, adding a small module for terminal decoration and tuning it depending on variable, one can conceive that people may want the messages to not be decorated at all (for example in CI, or when writing to a file).
But, I don't see how red/green is not just basically an improvement.

PS: That being said, I do agree with @ulugbekna, error messages could be made better in general, which would be more beginner friendly, and would avoid discussions like this one ocaml/ocaml-lsp#404 (comment). But hey, you can't both have the best programming language AND the best error messages, I understand priorities

@gasche
Copy link
Copy Markdown
Member

gasche commented Apr 13, 2021

I haven't heard any fundamental disagreement to using colors, but I don't hear any consensus about which colors to gave. I disagreed with the specific rationale given by @EduardoRFS to use red/green. Many other suggestions have been made, for example same-color-as-the-location-marker for the current type and bold for the other type (from @dbuenzli, one of the most principled suggestions). If there was a choice that manages to gather a consensus, thanks to a convincing argument in favor, I think we would go for it.

Improving the error messages is definitely something we welcome. Various maintainers have worked on this in the past (and it's quite a bit of work), and everyone else is welcome to contribute improvements. In particular, if someone would like to turn @ulugbekna's suggestion into a PR, feel free -- of course it would then be discussed quite a bit, etc.

@gasche
Copy link
Copy Markdown
Member

gasche commented Jun 7, 2021

@EduardoRFS gentle ping: would you interested in proposing a first PR that uses bold for general type display in error messages?

@EduardoRFS
Copy link
Copy Markdown
Contributor Author

@gasche yes, this weekend I will be working on OCaml PRs

@XVilka
Copy link
Copy Markdown
Contributor

XVilka commented Sep 18, 2021

There is a label "error-messages", I suggest to assign it to this PR, it will finding all these error message improvements easier:
error-messages

@EduardoRFS EduardoRFS force-pushed the colorful-error-messages branch from 94dc5fd to 4abd7ca Compare February 2, 2023 18:06
@EduardoRFS
Copy link
Copy Markdown
Contributor Author

@Octachron @gasche let's go at least with bold for now, I changed the name to clash. Probably this is not extensive but it covers simple type clashes and module inclusion mistakes.

Unrelated question, but does the back tick closed by single quote makes sense?

The type `t' is required

type clash
module clash

let missing_field ppf item =
let id, loc, kind = Includemod.item_ident_name item in
Format.fprintf ppf "The %s `%a' is required but not provided%a"
Format.fprintf ppf "The %s `@{<clash>%a@}' is required but not provided%a"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The quotation ``...'should be replaced by the tag. There is also the issue that this is not aclash` but a `missing` item. If we use semantics tags rather than formatting tags, I think it is better to be clear on the semantics of each tags.

@gasche
Copy link
Copy Markdown
Member

gasche commented Feb 16, 2023

My gut feeling is that any inline code fragment displayed in the middle of prose deserves to be emphasized by using bold, so using code or inline_code for the tag name would be fine -- even if we don't consistently use the tag on all such situations yet.

On the other hand, I'm not sure that we want it for code that is not inline but in a block (on its own line with indentation). We could use two tags, code_inline and code_block, yet use bold for both for now. This leaves the door open to easily changing this in the future -- possibly just as the current discussion evolves.

@gasche
Copy link
Copy Markdown
Member

gasche commented Feb 16, 2023

Also: thanks a lot @EduardoRFS for coming back to this PR, which has the potential for a much-appreciated usability improvement for OCaml error messages.

@EduardoRFS
Copy link
Copy Markdown
Contributor Author

Agreed with @gasche and it is a more flexible semantics. Will be changing the PR in the following days.

Also trying to understand more about the error system, fundamentally I think we need better error engine to make changes like a one liner.

@Octachron
Copy link
Copy Markdown
Member

We ended up going with a more generic inline_code tag in #12210 .

It might make sense to revisit the behavior for mismatching error messages, but I moving to close this PR since its current implementation is a subset of #122210 .

Don't hesitate to reopen this PR @EduardoRFS if you have other plans however.

@Octachron Octachron closed this Jun 20, 2023
@Octachron
Copy link
Copy Markdown
Member

Thanks a lot everyone for the discussion that helped settle on the simplified and incremental design for #12210 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.