Skip to content

Warnings emitted by the compiler may contain \r\r\n on Windows (MPR#7106)#370

Merged
damiendoligez merged 2 commits intoocaml:trunkfrom
dra27:normalise-warnings
Jan 27, 2016
Merged

Warnings emitted by the compiler may contain \r\r\n on Windows (MPR#7106)#370
damiendoligez merged 2 commits intoocaml:trunkfrom
dra27:normalise-warnings

Conversation

@dra27
Copy link
Copy Markdown
Member

@dra27 dra27 commented Dec 23, 2015

dra27 referenced this pull request in dra27/ocaml Dec 23, 2015
Stray `\r` characters included in the attribute when parsed on Windows.
Fixed by specifying that deprecated_module.mli must be checked out with LF
endings.
@alainfrisch
Copy link
Copy Markdown
Contributor

Thanks! Since this is not performance critical, it seems the code could be made much simpler by using a single pass, creating the result with Buffer.add_char.

@mshinwell mshinwell changed the title PR7106 Warnings emitted by the compiler may contain \r\r\n on Windows (MPR#7106) Dec 23, 2015
@dra27
Copy link
Copy Markdown
Member Author

dra27 commented Dec 23, 2015

Yeah, it was one of those things that seemed like a good idea when I started writing it! Given that I've almost certainly written that function 2 or 3 times before, can I make a case for it usefully being in the standard library (and therefore preferring the two-scan but only one copy rather than single-scan, two copy version with Buffer?). I'm happy to add the relevant test cases to the test suite, if so...

@gasche
Copy link
Copy Markdown
Member

gasche commented Dec 23, 2015

It's delicate to change the standard library because there is no clear consensus on where we want to go with it, and the current peace treaty ("only the obvious functions") does not justify adding this one. Feel free to move this to utils/misc.ml, and add testsuite coverage for it.

@dra27
Copy link
Copy Markdown
Member Author

dra27 commented Dec 23, 2015

I didn't know about that module - I'd incorrectly thought that being used in the compiler made it a potential candidate! I'll move it there and add tests in tests/utils

@damiendoligez
Copy link
Copy Markdown
Member

First of all, this isn't solving the real problem, it's just sweeping the dust under the rug.

The real problem is not \r\n in warnings, it's naked newlines in string constants, whose semantics changes when git translates \n into \r\n. Programs with variables semantics are bad news, especially when the change is invisible.

The problem was solved a few years ago for normal strings by introducing warning 29. This solution obviously doesn't apply to quoted strings. Any ideas?

Then, if you want to fix it at the warning level, I have two remarks:

  • Your solution is way too complex. Just remove all \r with this function:
    let remove_cr s =
      let b = Buffer.create 80 in
      for i = 0 to String.length s - 1 do
        if s.[i] <> '\r' then Buffer.add_char b s.[i]
      done;
      Buffer.contents b
  • You shouldn't apply remove_cr to all those warnings that have no chance of containing a \r, but
    use it only in the Deprecated case of message. It makes it a bit clearer where these \r might come from.

You could also simply fix the test by using a normal string as the annotation payload.

@dra27
Copy link
Copy Markdown
Member Author

dra27 commented Dec 23, 2015

OK, I was under the impression from dra27@3985349#commitcomment-15122153 that the inclusion of the \r\n was not considered a problem. Normalising the line endings in the parsing of attributes seems more logical to me.
Fair enough for simply removal all \r - I was concerned as to whether the message is (or might be) used elsewhere where the translation might not be wanted.

@alainfrisch
Copy link
Copy Markdown
Contributor

that the inclusion of the \r\n was not considered a problem

I was only describing the current state.

Normalising the line endings in the parsing of attributes seems more logical to me.

The problem is for quoted strings, where we expect a byte-level correspondence between whatever is in the source code and what is kept in the AST.

@dra27
Copy link
Copy Markdown
Member Author

dra27 commented Dec 23, 2015

Sorry, the way I phrased that was unnecessarily defensive. There are two options that "fix" the test and therefore truly sweep up the dust:

  1. Declare deprecated_module.mli as needing Unix line-endings in .gitattributes (which was my original solution which opened this) - potentially good inasmuch as it records the problem in a commit
  2. Change the message to be one line only. Is there a reason why it was a multiline message originally, though?

Both options dismiss this PR and hopefully expedite the merging of #366 (which is what I'm originally after...). Does that the leave the MPR as something that wants fixing at some point?

@damiendoligez
Copy link
Copy Markdown
Member

Fair enough for simply removal all \r - I was concerned as to whether the message is (or might be) used elsewhere where the translation might not be wanted.

The message is only to be used for display, so I really think the simple solution is better.

@damiendoligez damiendoligez added this to the 4.03.0 milestone Dec 28, 2015
@dra27
Copy link
Copy Markdown
Member Author

dra27 commented Dec 28, 2015

OK - it shouldn't ever be relevant here, but my thoughts/over-engineering are probably influenced by porting work in OPAM, where \r is used for its true meaning (i.e. terminal carriage return) and the simpler solution would be semantically wrong!
As you've added to it the 4.03.0 milestone, I take it you'd like the message fixed as requested, rather than altering the test suite to "mask" the issue instead?

@damiendoligez
Copy link
Copy Markdown
Member

By adding it to the 4.03 milestone, I just mean I think it should/will be resolved before 4.03.0. I agree altering the test suite is not a satisfactory solution, and I won't fight for the simple version of CR removal.

I'll merge this PR as soon as you:

  1. move normalise_eol to misc.ml.
  2. add a comment to Warnings.print to explain why we need to call normalise_eol.

@dra27 dra27 force-pushed the normalise-warnings branch from ac098c2 to b5e1a49 Compare January 4, 2016 12:46
@dra27
Copy link
Copy Markdown
Member Author

dra27 commented Jan 4, 2016

Happy New Year! Changes made. I apologise for creating the appearance of a fight - it's definitely your project (which I don't mean in the double-meaning that various guides as to what the British say vs. what they mean might suggest!) - I was going to put remove_cr in as suggested, which I'm quite happy to do... the conversation reveals that I should learn to put more comments as to why I've done something, as opposed to what, in my patches (which I've now done both in utils/misc.mli and also before the call to Misc.normalise_eol in utils/warnings.ml

utils/misc.mli Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: subsequently.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😊 Really should install full vim in the VM so I have the spell-checker...

@dra27 dra27 force-pushed the normalise-warnings branch from b5e1a49 to e641061 Compare January 4, 2016 13:12
@dra27 dra27 force-pushed the normalise-warnings branch from e641061 to 726b47e Compare January 15, 2016 22:42
@alainfrisch
Copy link
Copy Markdown
Contributor

There are more important topics, but what about doing the simpler version of removing all \r characters, but only as a rewriting for the payload string of the deprecated attribute?

dra27 added 2 commits January 18, 2016 16:19
The formatters used for printing warnings have text mode translation
enabled which means that any Windows endings which creep into warning
texts (e.g. from attributes) result in \r\r\n in the output.

The effect is largely innocuous, except that it causes the
deprecated_module_use test to fail on Windows.
The formatters used for printing warnings have text mode translation
enabled which means that any Windows endings which creep into warning
texts from deprecated attributes result in \r\r\n in the output.

The effect is largely innocuous, except that it causes the
deprecated_module_use test to fail on Windows.
@dra27 dra27 force-pushed the normalise-warnings branch from 726b47e to 469810f Compare January 18, 2016 16:42
@dra27
Copy link
Copy Markdown
Member Author

dra27 commented Jan 18, 2016

Rebased with a second commit to show the changes - up to you if you would like it squashed or merged as two.

damiendoligez added a commit that referenced this pull request Jan 27, 2016
PR#7106: Warnings emitted by the compiler may contain \r\r\n on Windows
@damiendoligez damiendoligez merged commit 6e6cabf into ocaml:trunk Jan 27, 2016
@damiendoligez
Copy link
Copy Markdown
Member

@dra27 thanks for putting up with our nitpicking :-)

@dra27 dra27 deleted the normalise-warnings branch February 8, 2016 09:23
@dra27 dra27 mentioned this pull request Oct 3, 2017
anmolsahoo25 pushed a commit to anmolsahoo25/ocaml that referenced this pull request Aug 25, 2020
mshinwell added a commit to mshinwell/ocaml that referenced this pull request Mar 26, 2021
chambart added a commit to chambart/ocaml-1 that referenced this pull request Jan 4, 2022
EmileTrotignon pushed a commit to EmileTrotignon/ocaml that referenced this pull request Jan 12, 2024
* fixing title case

* fixed another title case header

Signed-off-by: Christine Rose <professor.rose@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants