Skip to content

Added a SIGSEGV handler that dumps the stacktrace to ease reporting#10124

Merged
akien-mga merged 1 commit into
godotengine:2.1from
marcelofg55:handle_sigsegv
Sep 21, 2017
Merged

Added a SIGSEGV handler that dumps the stacktrace to ease reporting#10124
akien-mga merged 1 commit into
godotengine:2.1from
marcelofg55:handle_sigsegv

Conversation

@marcelofg55

@marcelofg55 marcelofg55 commented Aug 6, 2017

Copy link
Copy Markdown
Contributor

This PR adds a simple crash reporter that dumps the stacktrace to stderr, implemented for various OSes:

  • On Linux it adds a signal for SIGSEGV/SIGFPE that prints the stacktrace when the signal is triggered. It uses addr2line to get file/line number.

  • On OS X it also adds a signal for SIGSEGV/SIGFPE that prints the stacktrace when the signal is triggered. It uses atos to get file/line number.

  • On Windows it uses __try/__except to catch exceptions. Only supported on MSVC (no mingw support yet).

Note, the code is only enabled with crash_report=yes on debug mode, also after the stacktrace is generated the crash is passed to the OS so that you can debug if necessary.

This is a screenshot of how it shows the stacktrace on Linux:
screenshot from 2017-08-06 17-18-12

On Windows:
crashreport

On OS X:
screen shot 2017-08-07 at 1 36 55 pm

@reduz

reduz commented Aug 6, 2017 via email

Copy link
Copy Markdown
Member

@marcelofg55

Copy link
Copy Markdown
Contributor Author

Yeah, usually Linux users do know how to use gdb and such but I've seen many crash Issues reported without a stacktrace, so I thought this could help users to easily report a crash with the stacktrace. I agree that conditions to reproduce are usually more helpful to solve the crashes, but with a stacktrace included with the Issue report it lets you know where the problem might be.

@reduz

reduz commented Aug 6, 2017 via email

Copy link
Copy Markdown
Member

@Zireael07

Copy link
Copy Markdown
Contributor

I agree that it's easier to set up gdb on Linux, but a crash reporter of some kind for Windows would be lovely (aaargh gdb and Windows aaargh)

@marcelofg55

marcelofg55 commented Aug 6, 2017

Copy link
Copy Markdown
Contributor Author

Lines of code seems possible on Linux, need to work it out.
Edit: I'm not sure how crash reporters work on other apps, but I'll look into it.

@marcelofg55

Copy link
Copy Markdown
Contributor Author

I was just looking into how other apps handle crashes and for example I've found that unreal has this type of crash reporter, seems pretty cool in my opinion: https://answers.unrealengine.com/storage/attachments/41297-ue4_crash.png

@marcelofg55

Copy link
Copy Markdown
Contributor Author

Update: I managed to improve the stacktrace output on Linux, now the function name is demangled and the file/line number is shown as well:

screenshot from 2017-08-06 17-18-12

@marcelofg55 marcelofg55 force-pushed the handle_sigsegv branch 4 times, most recently from 44b745f to d655d9a Compare August 6, 2017 20:45
@bruvzg

bruvzg commented Aug 6, 2017

Copy link
Copy Markdown
Member

@marcelofg55 atos -o execpath addr should give line number on macOS in the same manner as addr2line do on Linux

@marcelofg55

Copy link
Copy Markdown
Contributor Author

@bruvzg thanks for the tip, I'll look into it

@neikeq

neikeq commented Aug 6, 2017

Copy link
Copy Markdown
Contributor

@reduz Godot binaries built with mono also have a SIGSEGV handler from mono that dumps the stacktrace, this is an example report: https://gist.github.com/neikeq/5002fd23f9d6d2f3843257d9326bcb42

@marcelofg55

Copy link
Copy Markdown
Contributor Author

Another update, been working on the windows code for this, here's how the crash report is looking so far:
crashreport

@marcelofg55 marcelofg55 force-pushed the handle_sigsegv branch 2 times, most recently from be602a6 to 93a0004 Compare August 7, 2017 04:45
@akien-mga

Copy link
Copy Markdown
Member

This is awesome! :)

If you need references for cross-platform (especially Windows) stack traces, you can check:

The code is GPL, but I know the author and I'm positive he'd agree to have us draw ideas from it/reuse some chunks under the MIT, if needed. (Haven't checked your code yet, maybe you already have a better feature ;)).

@marcelofg55

Copy link
Copy Markdown
Contributor Author

Thanks for the references @akien-mga! The current implementation on this PR lacks mingw support yet so I'm going to check how the Open Dungeons code works for that one, and I'll check the others for ideas as well :).

@akien-mga

Copy link
Copy Markdown
Member

The code is GPL, but I know the author and I'm positive he'd agree to have us draw ideas from it/reuse some chunks under the MIT, if needed. (Haven't checked your code yet, maybe you already have a better feature ;)).

CC'ing the author @hwoarangmy (hi! how are you doing? :D) for confirmation that we can use his GPL'ed code as reference for a MIT implementation.

@marcelofg55

Copy link
Copy Markdown
Contributor Author

Yet another update, this time with the feature for OS X (it uses atos to get file/line number as suggested by @bruvzg):
screen shot 2017-08-07 at 1 36 55 pm

@marcelofg55

Copy link
Copy Markdown
Contributor Author

Sure, I've just changed the message.

@akien-mga

Copy link
Copy Markdown
Member

I really like this feature but I think it's better to remove the "please include this when reporting" part considering this might be shipped with actual games, and they may have different bug report policies.

Well, if the backtrace is shown in exported games, I would expect users to include it when reporting it to the game developer. The developer can then decide if it's meaningful or not to make an upstream bug report - to the gamer, it should not matter that it's a crash of Godot, but of their game.

So the proposed message was fine IMO. There could be a project option to define a bug report URL / email address, so that you can make the message something like:

@akien-mga

Copy link
Copy Markdown
Member

The main concern @reduz had about this feature is that if you catch SIGSEGV, it might make it cumbersome to really debug issues with gdb. Can you confirm that this would be an issue, or would gdb have the priority and catch the segfault before your logic gets triggered?

I personally think this feature would be nice to have by default on release builds (both for the editor and the template), so if the above concern is valid, I would propose this logic:

  • add a command line switch (in main.cpp) to enable/disable the feature
  • in tools=yes target=release_debug and tools=no target=release (release builds of the editor and templates, i.e. the binaries we distribute), enable the feature by default - it can be disabled via the command line switch if needed
    • before doing that, it would be good to assess how useful the backtraces are in such binaries though, as we strip some of the debug symbols
  • in tools=yes target=debug and tools=no target=release_debug (debug builds of the editor and templates), disable the feature by default so that they can be debugged via gdb/windbg - it can be enabled via the command line switch if needed

@marcelofg55

Copy link
Copy Markdown
Contributor Author

@akien-mga @reduz I've just tested on OS X and Linux, and on both cases gdb has priority over the handle_sigsegv code, so the function doesn't get in the way. So I guess that the command line switch is not needed then?

@marcelofg55

Copy link
Copy Markdown
Contributor Author

As suggested by @akien-mga I've added another option to scons for the crash message using crash_report_message="message here" (defaults to "Please include this when reporting the bug on https://github.com/godotengine/godot/issues")

@akien-mga

Copy link
Copy Markdown
Member

As suggested by @akien-mga I've added another option to scons for the crash message using crash_report_message="message here" (defaults to "Please include this when reporting the bug on https://github.com/godotengine/godot/issues")

I wanted this as a project-specific option actually (in ProjectSettings), so that users don't have to recompile their export templates just for that. If that's not possible (e.g. ProjectSettings not initialized at that stage), it's probably better to drop the option altogether and use a generic message. Our compiler defines are way too long already :)

@akien-mga

akien-mga commented Aug 11, 2017

Copy link
Copy Markdown
Member

So if it does not conflict with normal debugging, I would just make this enabled by default (without scons option to disable it, so that we don't have to pass around a compiler define forever). Then I'd add an option to the Godot binary (via main.cpp) to disable it when needed (e.g. -disable-backtrace).

@marcelofg55 marcelofg55 force-pushed the handle_sigsegv branch 3 times, most recently from c93503b to 28119fe Compare August 11, 2017 21:39
@marcelofg55

Copy link
Copy Markdown
Contributor Author

So I've added that -disable-backtrace and the project setting for the message as well. And removed the scons previous stuff.
By the way, this is tagged as 3.0 but I've made the PR for the 2.1 branch (I will port these changes for the master branch after this gets approved of course).

@hpvb

hpvb commented Sep 12, 2017

Copy link
Copy Markdown
Member

Sorry, this is all fine for 2.1.

@hpvb

hpvb commented Sep 12, 2017

Copy link
Copy Markdown
Member

There are some concerns about adding all of these to the platform's main files.

Would you mind splitting out the implementations in each platform/ directory? Maybe have a platform/name/crash_handler.cpp/h and just call it from main?

Thanks! I really love this feature though.

@hpvb

hpvb commented Sep 13, 2017

Copy link
Copy Markdown
Member

Would you mind splitting the crash handler code in the 2.1 version also?

@marcelofg55 marcelofg55 force-pushed the handle_sigsegv branch 2 times, most recently from 0608318 to 7ae6b03 Compare September 14, 2017 20:24
@marcelofg55

Copy link
Copy Markdown
Contributor Author

@hpvb Just changed the commit with the crash handler .cpp/h files.

@hpvb

hpvb commented Sep 14, 2017

Copy link
Copy Markdown
Member

I think we're still discussing what we want to do for 2.1.

@akien-mga

Copy link
Copy Markdown
Member

Are some fixes needed based on early feedback in the master branch? (some compiles fixes + preventing blocking the debugger on MSVC)
I'll merge once those are in.

@marcelofg55

Copy link
Copy Markdown
Contributor Author

Just added those changes.

@akien-mga

Copy link
Copy Markdown
Member

Thanks, I'll merge once Travis has finished building.

@akien-mga akien-mga merged commit 1391269 into godotengine:2.1 Sep 21, 2017
@marcelofg55 marcelofg55 deleted the handle_sigsegv branch September 21, 2017 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants