Original bug ID: 7185
Reporter: @mlasson
Assigned to: @damiendoligez
Status: assigned (set by @mshinwell on 2016-12-08T09:13:56Z)
Resolution: open
Priority: low
Severity: feature
Category: runtime system and C interface
Monitored by: @nojb @gasche @diml @ygrek @dbuenzli @alainfrisch

Bug description

As you all know, there are essentially two ways a caml program can fail due to a lack of memory.

Either,

• it raises a catchable Out_of_memory exception; this is usually raised by a C binding that wants to report an allocation failure or by the runtime when caml_alloc_shr while we are not in a minor collection,

• or with a fatal error that terminates the program if caml_alloc_shr is not able to expand the heap in the middle of a minor collection.

The first situation can be exercised by allocating big chunks of memory:
let _ = try
ignore (Array.init Sys.max_array_length
(fun k -> Array.init
Sys.max_array_length (fun _ -> k)))
with Out_of_memory -> Printf.printf "I've survived\n%!" (* Printed *)

whereas the second is likely to happen while reaching the limit with small increments of memory:
let rec init acc n =
if n >= 0 then init (n :: acc) (n - 1)
else acc

let _ = try
ignore (init [] max_int)
with Out_of_memory -> Printf.printf "I've survived\n%!" (* Not printed *)

In some applications reaching the memory limits is a normal way to use a program (eg. in scientific or financial computing it is natural to push a system to its limits and these limits are often quite difficult to estimate without actually running the computation). In that cases, having a decent way to report the failure and its reason to the user is more complicated in the case of fatal error.

Would it really be impossible to hack the GC to avoid fatal errors and always raise Out_of_memory ?
While we are in the middle of a minor collection, could we somehow undo the unfinished job of the GC (ie. put everything we've just copied in major heap back to the minor heap) in order to raise the exception as if we've never tried to collect ? Or has some information been definitively lost in the process ?