Status update: There are significant thread-related changes in ruby 3.x due to the introduction of ractors (ruby actors). This breaks several assumptions in rbspy when we try to look up the current execution context and use it to find stack frames. I've been slowly working through these issues and trying to fix them without making the version-specific code harder to reason about.

This code should currently work on Linux with ruby 2.7+. It builds but is broken (no stack frames) on macOS, which is next on my list to debug. I haven't tried Windows or FreeBSD yet.

It's able to find stack traces on macOS if I re-run bindgen on that platform, but then it's broken on Linux. A diff of the bindings turns up several changes but nothing directly related to our stack frame code (afaict). I wouldn't be surprised if there's a type size or alignment issue, though. I'll try to narrow it down, and if it can't be resolved then we may need to fork the bindings based on platform -- or find a way to avoid the problematic types.

This works on macOS now.

I tried it on Windows tonight and didn't have much luck. The tests pass, but it seems like we're looking at the wrong memory locations when we try to get stack frames. Makes me wonder if we'll need to run bindgen separately on each platform to get the correct types (like I've already done for macOS and Linux). The main sticking point, afaict, is that the pthread primitives are platform-specific and differ pretty significantly in their size (e.g. struct vs union), so it's easy to jump to the wrong location in memory if the types and platform are mismatched. We need to step over a few of those pthread types as we're walking through the ractor and threads structs to get the execution context. If anyone knows of a shortcut to get the EC, please let me know.

If anyone knows of a shortcut to get the EC

It's the opposite of a shortcut, but I found a way to get the EC address without needing to step over pthread types. The VM struct has a pointer to the current main thread struct, which in turn has a pointer to the EC.

It works on Linux, macOS, and Windows without requiring separate bindings for each platform, which is a relief. rbspy seems to be generally broken on FreeBSD 12 for me (can't get stack frames, even on master with 2.7.2), so I'm keeping the build healthy but otherwise not spending time on it.

EDIT: This approach works but always profiles the main thread, which isn't what we want. Next puzzle is to find a way to refresh the EC pointer.

@jvns When you have time, this is ready for a look please. The logic in the get_execution_context_from_vm macro is a bit hacky, but it's worked well for me so far, and it sidesteps the maintenance headache of needing to run bindgen on all platforms each time we add a new version of ruby 3+.

If we need to fall back to the bindgen approach (now or later), I think it would be good to invest some effort in automating all of it through GH Actions, maybe building on Igor's docker work, so that generating new bindings is as simple as running a CI job. (As I'm writing this, it seems like a good thing to do regardless, hah. :)

lgtm! This seems very reasonable to me and it feels less hacky than some of the other things we do in rbspy like is_maybe_thread (this follows way less pointers!). Thanks for the comments too :)

I'm so happy you did this!

This is in good shape now, I think. The Travis build gets through the tests and then fails when it uses libjemalloc, but I'm able to use libjemalloc and libtcmalloc locally with ruby 2.4. The malloc tests have been flaky in CI for me before.

If anyone has time to help test this, please do.

I'll probably merge tomorrow if there are no objections, and then cut a release next week.