Skip to content

fix: line endings are set based on file#339

Merged
AndrewSouthpaw merged 53 commits into
thlorenz:masterfrom
thompson-tomo:fix/#161_lineEndings
Apr 26, 2026
Merged

fix: line endings are set based on file#339
AndrewSouthpaw merged 53 commits into
thlorenz:masterfrom
thompson-tomo:fix/#161_lineEndings

Conversation

@thompson-tomo

Copy link
Copy Markdown
Contributor

Closes: #161
Closes: #216

Tool now detects what line endings have been used in the content and uses that in the generated toc to strive for consistency.

Blocked by: #337 & #338

Comment thread lib/transform.js Outdated
if (lf > crlf && lf > cr) return '\n';
if (cr > crlf && cr > lf) return '\r';

return '\n';

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this default be user definable? Cc @AndrewSouthpaw

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would ease some of what I mention above. Something like --line-ending-default or something, which we can fall back to (and not throw) in cases of mixed line endings.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's push configurable option to next minor.

Comment thread lib/transform.js Outdated
Comment on lines +129 to +131
if (crlf > lf && crlf > cr) return '\r\n';
if (lf > crlf && lf > cr) return '\n';
if (cr > crlf && cr > lf) return '\r';

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about how we should handle files with mixed endings. Perhaps we should do.

Suggested change
if (crlf > lf && crlf > cr) return '\r\n';
if (lf > crlf && lf > cr) return '\n';
if (cr > crlf && cr > lf) return '\r';
if (lf + cr === 0 && crlf > 0) return '\r\n';
if (crlf + cr === 0 && lf > 0) return '\n';
if (crlf + lf === 0 && cr > 0) return '\r';

Thoughts @AndrewSouthpaw

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm this could be a major change, as I mentioned here.

IMO it'd make sense for this to be behind a flag for v2, and change the default to automatically detect for v3.

For files with mixed line endings, I'm leaning towards throwing, because there's not really a reasonable thing to do. Whatever we choose, it could be the wrong behavior for the user, and quite subtle. We can give a flag for them to choose the override (below).

Printing a warning would be another option, but I'm generally a fan of failing fast, especially given that mixed line endings is probably an indication that something is weird and wrong about their file, and it'd be helpful for them to know about it.

Thoughts?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree #216 would be a major change especially as it uses the system default rather than detecting and it also re-writes the entire doc.

I think it would be ok if document is consistent use what is detected, otherwise use configured which has a default of '\n'.

This would mean:

  • \n docs already with upto date toc would always use \n
  • \r\n docs already with upto date toc would use config option with default \n. Can Result in mixed doc as per now if not configured otherwise.
  • \r\n docs already with toc containing normalised line endings via script would use \r\n, changed behaviour. This would be low likelihood but what user wants as it removes need to add in a normalisation process. Effectively a bug fix.
  • \r\n doc without toc would use \r\n, changed behaviour. In this case it is a change but avoids creating a mixed document which I see as a positive.
  • \n doc without toc would use \n, no changed behaviour.

So we would only get a chance in a few scenarios. Let me know if you are on the same page.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this some more, I feel we can merge this without the configurable default. That way this can be released as part of 2.4.1 due to it purely being a fix. This is because the only time a user would see a change is if they are post processing the toc to normalize the endings and all they would see is their processing would have no effect and for new docs we are avoiding introducing a line ending mismatch.

Comment thread lib/transform.js Outdated
Comment on lines +129 to +131
if (crlf > lf && crlf > cr) return '\r\n';
if (lf > crlf && lf > cr) return '\n';
if (cr > crlf && cr > lf) return '\r';

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm this could be a major change, as I mentioned here.

IMO it'd make sense for this to be behind a flag for v2, and change the default to automatically detect for v3.

For files with mixed line endings, I'm leaning towards throwing, because there's not really a reasonable thing to do. Whatever we choose, it could be the wrong behavior for the user, and quite subtle. We can give a flag for them to choose the override (below).

Printing a warning would be another option, but I'm generally a fan of failing fast, especially given that mixed line endings is probably an indication that something is weird and wrong about their file, and it'd be helpful for them to know about it.

Thoughts?

Comment thread lib/transform.js Outdated
Comment thread lib/transform.js Outdated
if (lf > crlf && lf > cr) return '\n';
if (cr > crlf && cr > lf) return '\r';

return '\n';

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would ease some of what I mention above. Something like --line-ending-default or something, which we can fall back to (and not throw) in cases of mixed line endings.

Comment thread lib/transform.js Outdated
@thompson-tomo thompson-tomo marked this pull request as ready for review April 15, 2026 11:05
@thompson-tomo thompson-tomo changed the title fix: line endings are set based on file #161 fix: line endings are set based on file Apr 17, 2026
@thompson-tomo thompson-tomo marked this pull request as draft April 21, 2026 05:43
@thompson-tomo thompson-tomo marked this pull request as ready for review April 22, 2026 00:21
@thompson-tomo

Copy link
Copy Markdown
Contributor Author

@AndrewSouthpaw now that #338 is merged this one is good for review.

@AndrewSouthpaw AndrewSouthpaw merged commit e15a1c7 into thlorenz:master Apr 26, 2026
5 checks passed
@thompson-tomo thompson-tomo deleted the fix/#161_lineEndings branch April 26, 2026 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unusable as 'pre-commit' hook, since it will always fail.

2 participants