Skip to content

Improved representation of JRuby for JVM tools#8259

Merged
headius merged 9 commits intojruby:9.5-devfrom
headius:better_debug_jit
May 29, 2024
Merged

Improved representation of JRuby for JVM tools#8259
headius merged 9 commits intojruby:9.5-devfrom
headius:better_debug_jit

Conversation

@headius
Copy link
Copy Markdown
Member

@headius headius commented May 24, 2024

This PR will encompass several improvements to JRuby's representation of code and objects, in order to make JVM tooling more useful.

Among the ideas explored here:

  • Cleaner representation of methods and class for stack traces and profiling.
  • Always-on class reification to show real class names in JVM heap inspection tools.
  • Better JIT configuration to allow JVM debugging tools to step through Ruby code and set breakpoints and such.
    • This may include some changes to force larger scopes to JIT so they can be debugged, even if this means the JVM cannot native JIT them.
  • Enhancements for JVM debuggers to better see into dynamic scopes, Ruby objects with instance variables, and core Ruby types with complicated internals.

@headius headius added this to the JRuby 10.0.0.0 milestone May 24, 2024
@headius
Copy link
Copy Markdown
Member Author

headius commented May 24, 2024

This piece may not make sense to introduce the dependency, but it's super cool... IntelliJ annotations for better rendering of objects in the debug view:

https://gist.github.com/headius/5c7e90fe501a66b0159a7bfecabf9907

With these changes, DynamicScope will show a simple name to value mapping for local variables, and RubyBasicObject will show instance variable names mapped to their current values.

This was discovered while investigating why the IntelliJ debugger can't currently render DynamicScope... it turns out they added a renderer for us years ago but it's based on an old object layout: https://youtrack.jetbrains.com/issue/IDEA-353990/JRuby-DynamicScope-renderer-is-broken

@headius
Copy link
Copy Markdown
Member Author

headius commented May 24, 2024

An additional change here makes a new class for not-given blocks, but might be risky since up til now we have not had any subclasses of Block. Note the change in the JIT necessary to make sure that NotGiven is cast to Block for indy purposes:

https://gist.github.com/headius/59b074be588004ade9e32d68843aff54

headius added 9 commits May 28, 2024 13:57
This improves how local variables look in JVM debuggers in the
following ways:

* Non-heap local variables show up as their original names, rather
  than the $t_NAME_# format.
* The "self" variable shows up as "self" rather than $self, since
  "self" is not a valid variable name in Ruby.
* The implicit block is currently named "&" but we may want
  something better.
* The dynamic scope is named "variableStore" to make it clear that
  is where the variables are in heap-based scopes.
* All other locals like the context, called method name, static
  scope, and temporary variables are no longer declared and do not
  show up in debugging.

This cleans up simple flat scopes in the debugger, making it much
easier to understand what values are in which variables.
It's at least easier to read!
This restructures the "mangled" JVM method name used for the JVM
stacktrace and mined for the Ruby backtrace to make it more
readable to normal users.

Instead of "RUBY$method$foo$1"

we get "❤ def foo #1"

The heart is the RUBY marker, def indicates a method, and the # is
the same scope numbering the JIT had before. The delimiter is a
unicode non-breaking space.

This shows in the JVM backtrace similar to these examples:

	at blah.️❤ def foo$1(blah.rb:2)
	at blah.️❤ def bar$2(blah.rb:6)
	at blah.️❤ {} go$6(blah.rb:17)
	at blah.️❤ def times$4(blah.rb:12)
	at blah.️❤ def go$5(blah.rb:17)
	at blah.️❤ {} \=\^main\_$0(blah.rb:20)
	at blah.️❤ def times$4(blah.rb:12)

But still can be parsed by the Ruby backtrace miner to produce the
proper Ruby backtrace elements.

The format of this is negotiable.
Naively declaring variables as existing from the beginning of the
method until the end does not seem to be a problem for the JVM,
but debugging tools like IntelliJ's JVM debugger do not appear to
like it. Instead, we add an additional map to track the first
assignment of each local variable and use that as the starting
offset. This leads to some additional labels in the bytecode, but
IntelliJ is now able to properly display all local variables in
its debugger.

Fixes jruby#8256
These names were hardcoded which broke aspects of compilation and
JIT testing.
@headius
Copy link
Copy Markdown
Member Author

headius commented May 29, 2024

Going to stop here but I've filed #8265 for propagating the relative path through to the JIT bytecode source attribute, if that turns out to be useful for debugging tools (see #6844 and some discussion in https://youtrack.jetbrains.com/issue/RUBY-32948/JRuby-DynamicScope-renderer-is-broken).

@headius headius merged commit a29afff into jruby:9.5-dev May 29, 2024
@headius headius deleted the better_debug_jit branch May 29, 2024 18:43
@JasonLunn
Copy link
Copy Markdown
Contributor

Amazing! Is there anything you need contributed for this effort?

@headius
Copy link
Copy Markdown
Member Author

headius commented Jun 16, 2024

@JasonLunn Hey I missed this but yeah there's probably a ton of ways to help! What JVM tools are you familiar with?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants