Rebased and testing this against 9.3. This will not make it into the 9.3.10 release due to the short turnaround time, but we will merge for 9.3.11 and 9.4.1. We will release 9.4.1 first to see if users run into any issues there.

9.3.11

Thank you @headius , really appreciate it.

The alternative to updating 9.3 to Psych 5.1 would be to do a similar SnakeYAML Engine conversion and release on the Psych 3.3 line

I don't think that's a good idea as older psych versions seem to contain a broken SnakeYAML version check. That check was only removed with the switch to snakeyaml-engine.

The only real justification for the upgrade is to eliminate the CVE-prone SnakeYAML. Even then, those CVEs are only flagged because the library is present in JRuby's distribution; we do not ever use the offending code and are not at any risk.

@apurtell: You are the only person to report issues with the older SnakeYAML. Is it possible for you to upgrade to JRuby 9.4.1 or higher?

Our community includes users that write administrative scripts to automate various actions using HBase's shell, which extends IRB. If the required Jruby upgrade moves up the Ruby language level and might break user scripts, our compatibility guidelines, specifically operational compatibility, will prevent us from taking it until we can make a future major release. Given the circumstances we could make an exception and do it in a new minor release. No new minor release is currently roadmapped.

If moving from 9.2.13 to 9.4.1 won't break our shell it could be fine. Was already worried about 9.2.13 -> 9.3.x, but we do have a functional shell after that change. I did a quick check using HBase 2.5, our current releasing code line, with 9.4.1.0 (and joni 2.1.48 and jcodings 1.0.58), and we do not have a functional shell, there is some kind of problem with irb and reline:

io/console on JRuby shells out to stty for most operations
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.5.3-SNAPSHOT, rUnknown, Thu Mar  2 23:02:56 UTC 2023
Took 0.0010 seconds
NoMethodError: undefined method `gsub' for nil:NilClass
    check_multiline_prompt at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/reline/line_editor.rb:124
                   map at org/jruby/RubyArray.java:2812
    check_multiline_prompt at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/reline/line_editor.rb:124
              rerender at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/reline/line_editor.rb:441
        inner_readline at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/reline.rb:319
         readmultiline at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/reline.rb:254
         readmultiline at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/forwardable.rb:238
         readmultiline at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/forwardable.rb:238
                  gets at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:418
            eval_input at uri:classloader:/irb/hirb.rb:95
         signal_status at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:770
            eval_input at uri:classloader:/irb/hirb.rb:94
                   lex at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:284  
  each_top_level_statement at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:253
                  loop at org/jruby/RubyKernel.java:1586
  each_top_level_statement at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:250
                 catch at org/jruby/RubyKernel.java:1292
  each_top_level_statement at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:249
            eval_input at uri:classloader:/irb/hirb.rb:111
                   run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:491
                 catch at org/jruby/RubyKernel.java:1292
                   run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:490
                <main> at classpath:/jar-bootstrap.rb:225

@apurtell Ok I understand your situation. I would go ahead with this upgrade but I still have concerns about making a major version change to the YAML support in a minor update to 9.3.

I would like you to open a bug for the shell issues you mention, so we can try to improve the situation there and make an upgrade more feasible. I will also chat with @enebo and other contributors about the potential impact of these major updates to Psych and SnakeYAML in 9.3.x.

What's the expected timeline for this issue and the 9.3.11 release? My understanding is that this change is necessary to address this recent vulnerability: GHSA-mjmj-j48q-9wg2

@seanstory as the comments above and at #7570 (comment) note, JRuby is not actually vulnerable to this and such deserialization vulnerablities so the technical risk and effort for JRuby folks needs to be weighed against the limited value of reducing noise from simple scanning tools which are unaware of the context by which a library is actually used.

Assuming your only use of Snakeyaml is via JRuby and Psych you'll be at no lower real risk if this update is merged, while the JRuby maintainers risk breaking other behaviour from the major version update of SnakeYaml.

My personal guess would be that overall effort is better spent moving to JRuby 9.4, which presumably you'd want to be doing anyway if you're concerned about security given Ruby 2.6/2.7 are EOL (and thus many related libraries may also start dropping Ruby 2.6/2.7 support and go unmaintained for security purposes, even if JRuby 9.3 stays supported for a while longer)

Thanks, @chadlwilson, I hadn't found #7570, but stumbled to this PR directly.

Yep, we'd come to the same conclusion that we weren't actually vulnerable, and are just looking for when we can get this removed from popping up on scans. I should have phrased my comment differently. I'm not looking to apply urgency, just trying to be forward looking for when we might be able to assure our customers that this will drop off their scans.

We are slowly working towards a 9.4 upgrade, but that is a more involved process than upgrading to 9.3.11 would be for us. So depending on timing, we may go to 9.3.11 first.

Fair enough @seanstory . The transitive nature of noise from scans is completely unsustainable in our industry right now so I feel your pain. My attempt to support this in the open source world (for GoCD in this case) is to document the suppressions (FPs, product not actually vulnerable) in a form that users can either consume themselves or refer to and transplant to their own scan tool.

Hi,

I'm also affected by this SnakeYAML CVE - I also don't think it affect my app, but it is required to update because of the security scan. I am unable to update to 9.4.x or higher, because I cannot upgrade to Ruby 2.7+.

Any timelines for merging this into 9.3.x? It seems like it hasn't been done yet based on this issue history.

At this point I am leaning strongly toward not changing the default psych we ship in JRuby 9.3.x. There are issues in upgrading to a YAML 1.2 engine, as reported by users, and I'm not comfortable with making a breaking change in a minor release. It Is possible to forcibly downgrade to a lower installed psych version, but many people will not realize that and in some cases may not control the list of dependencies of some third-party library or app.

Example issues likely due to YAML 1.2:

• Prevent snakeYAML add backslash when dump multiple line string ruby/psych#642
• Expose load settings ruby/psych#613 (comment)

The CVEs being reported are dubious, but even if they are valid the relevant functionality in SnakeYAML is not used by JRuby at all. So it's annoying that we get flagged, but it's not actually a security risk.

I am willing to consider other options here, because I sympathize with the CVE issues many of you are having. I would also note that we are likely to stop maintenance of 9.3 at the end of this year and 9.4 has stabilized quite well, so upgrading to 9.4.x is currently your best choice.

All options that I see:

• Upgrade psych. This seems too risky given the leap to YAML 1.2 as described above.
• Unbundle psych altogether. Probably not a realistic option since it's part of the standard library.
• Release a separate jruby-complete artifact that only upgrades psych. This would solve the main CVE case of psych bundled within the release artifact, but it would be complicated to create and support.
• Do nothing. Users should upgrade to 9.4 if they have SnakeYAML CVE issues with 9.3.

@enebo How do you feel about this?

So the tradeoff is security scripts will tag us as unsafe even though we do not use the code in question vs introducing significant changes in YAML/SnakeYAML which will potentially cause other users to have issues. Based on past history I think this is a bad tradeoff. I do no think we should merge this into 9.3.

tag us as unsafe even though we do not use the code

Yeah so it's only the inconvenience (which is a significant inconvenience, to be sure) of having security tools flag us... but with no actual security risk to users.

If users having issues with security tools need help upgrading to 9.4, please let us know what we can do!