-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Segfaults on exit when the sigaltstack is changed by the program #11489
Description
The issue
If the user's program changes the sigaltstack (through a C library, for example) with the stack overflow detection active (HAS_STACK_OVERFLOW_DETECTION is defined), the program will crash with an invalid free when exiting.
The source
The function (caml_)setup_stack_overflow_detection calls sigaltstack with a stack placed from a malloc, that is then supposedly freed in caml_stop_stack_overflow_detection.
This function only checks if the flags indicates that the sigaltstack was disabled, and if not, frees it.
However, this is not valid in general, when other libraries may want more memory space (in fact, Wasmer, which is a Rust library, requires more than 8k, instead allocating 64k of stack space). In this case, the library changes the sigaltstack (with an enabled stack), but using a different method (for example, an mmap). Then, OCaml detects the sigaltstack is enabled, ad tries to free it.
The solution
The solution would be to remember the ss_sp variable (that was malloc'ed), then when freeing, we also need to check if this ss_sp was changed. If not, the case I just described did not happen, and we can free it.
Otherwise, we left everything as-is (because the external library might restore the stack and/or use it for its own purpose), potentially leading to a memory leak.
Affected version
Version 4.10 is guaranteed to fail, but the incriminating piece of code stays up until version 4.14.
At first glance, it does not seems like version 5.0 is affected (but I may be wrong, I have not tested it).