Consider the following example:
require "regexp_parser"
ascii_escaped = <<~'REGEXP'
[\.][b][\.]
REGEXP
utf8_escaped = <<~'REGEXP'
[\(][b][\(]
REGEXP
def loc(node, index)
[node.expressions[index][0].ts, node.expressions[index][0].te]
end
ascii_parsed = Regexp::Parser.parse(ascii_escaped)
utf8_parsed = Regexp::Parser.parse(utf8_escaped)
pp ["ascii", loc(ascii_parsed, 0), loc(ascii_parsed, 1), loc(ascii_parsed, 2)]
pp ["utf8", loc(utf8_parsed, 0), loc(utf8_parsed, 1), loc(utf8_parsed, 2)]
It has the following output, printing start/end locations for the content of each character class:
["ascii", [1, 3], [5, 6], [8, 10]]
["utf8", [1, 3], [7, 8], [10, 12]]
Notice how the locations are different. Both regexp have the same source length (12 in this case), yet they report different locations for the content of the character classes. More specifically, if you append more utf8-escaped character classes you get locations that are outside the length of the regexp.
If I switch out the character classes for a capture group, the locations are correct again.
Consider the following example:
It has the following output, printing start/end locations for the content of each character class:
Notice how the locations are different. Both regexp have the same source length (12 in this case), yet they report different locations for the content of the character classes. More specifically, if you append more utf8-escaped character classes you get locations that are outside the length of the regexp.
If I switch out the character classes for a capture group, the locations are correct again.