Yay!

👍

What happens if there are blank lines in the here doc (not necessarily with leading whitespace) ?

@Fryguy currently those would be considered lines with no indentation, so they would cause the entire heredoc to be flush left. That means that the documentation I just pushed is incorrect, but before I fix it, do you think it's better to ignore blank lines, or treat them as lines with no indentation?

I was originally thinking ignore them for the purposes of figuring out the strip size. As a user of the method, my least surprise would be with this:

class FancyHello
  def self.hello
    puts <<~README.inspect
      Hello

        World!
    README
  end
end

FancyHello.hello # => "Hello\n\n  World!\n"

Not 100% sure though...what do others think? @avdi?

With this last commit, lines which are blank (empty or consisting only of tabs and spaces) will not be used to find the base indentation level. On a blank line, any amount of indentation less than the heredoc's base indentation level will be ignored, while any additional indentation will be preserved.

I expect that literally written spaces/tabs would be stripped, but not escaped ones, such as \, \s, \t, \040, \x09, and so on.
If we use your approach, line_indent should be counted at parsing each lines, but not after the whole here doc, I think.

@nobu \ (backslash space) is now preserved when it appears at the start of a line (other escape sequences should be the same). To achieve this, I moved the counting of line_indent inside parser_tokadd_string. Is this more or less what you had in mind?

I've thought about this a lot, and I've ended up with two options that I would find acceptable:

#1. Indent is based on shortest-indented non-whitespace line. So:

class FancyHello
  def self.hello
    puts <<~README.inspect
      Hello

        World!
    README
  end
end

outputs:

Hello

  World!

#2. Final indent is based on the indent level of the closing marker (README, in this example). Output would be:

  Hello

    World!

Of the two, I suspect #1 is less likely to surprise people. In both cases, blank lines are ignored for the purpose of indent.

@avdi Seems like we're all leaning toward #1, that's the behavior implemented in this PR.

Escaped spaces seem fine.
Still not working well with string interpolation, #{}.
You'll need to reset heredoc_indent at the beginning of a heredoc but not for each fragments, as well as lex_strterm, and dedent them at the rule string1.
Also, heredoc_indent needs to be saved/restored around compstmt in string_content.

BTW, it's better to adopt the existing coding style (indent, braces, etc.) to send patches, even if it is far from your favorites.
This is not MUST and won't be the only reason to reject for ruby usually, but recommended in general.

@nobu I definitely didn't intend to introduce style inconsistencies! I guess you specifically meant where I was using just spaces for indentation, instead of tabs and then spaces ... if so, I think I have fixed it with this last commit. I'll work on the other issues later this week. Thanks for all the feedback!

@nobu With the changes you mentioned above, interpolation seems to be working now. I also updated ripper to provide the dedented string and added support for backticks.

👍 This would solve a long-time annoyance I (and presumably many others) have had with the heredoc syntax. Rails has had a solution to this for a while, but for non-rails code the need to remove indentation from heredoc strings has been rather irritating.

Hi! I'm guessing this would need a rebase if it was to be merged, but... is it still being considered? I would personally find it very handy to have it in core.

@bjmllr thank you so much for implementing this patch. I gave up a while back since I don't know C and lexer so well and couldn't finish the patch.

Per @avdi's comment, the original intention is to have the output as example #1 as well. I'm glad that this is being adopted.

I rebased this branch and made a first attempt at @matz 's request regarding the handling of hard tabs. It should now do something sensible for any indentation other than spaces followed by tabs on a single line.

The build error seems to be unrelated, something in test_fork.rb?

@bjmllr I re-runned Travis CI.

I'm confused. Why are tabs being treated as equivalent to spaces at all? E.g. If I write:

def hello
  puts <<~README.inspect
<tab>Hello

<space><space><space><space><space><space><space><space>World!
    README
  end
end

Are you saying that should be accepted by the compiler? Why? Why should that be any less invalid than:

def hello
  puts <<~README.inspect
<tab>Hello

<space><space><space><space>World!
    README
  end
end

or

def hello
  puts <<~README.inspect
<space><space><space><space>Hello

<space><space>World!
    README
  end
end

Shouldn't we just throw an error in all of those cases? Is there ever a legitimate reason why you'd want to allow inconsistent indentation in one of these blocks? What happens when someone has their editor set to display tabs as 4 spaces, and writes:

def hello
  puts <<~README.inspect
<space><space><space><space>Hello

<tab>World!
    README
  end
end

Why should that result in:

hello #=> "    Hello\n\n\tWorld!"

I certainly wouldn't expect that result intuitively. In such a case, wouldn't a well-written error message explaining that I'm mixing tabs and spaces be much more helpful for me as a developer?

Closing this since the feature was added in 9a28a29