Codecov Report

❗ No coverage uploaded for pull request base (master@37dbbdb). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master     #944   +/-   ##
=========================================
  Coverage          ?   94.65%           
=========================================
  Files             ?       45           
  Lines             ?     6623           
  Branches          ?        0           
=========================================
  Hits              ?     6269           
  Misses            ?      354           
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 37dbbdb...5cd1bfe. Read the comment docs.

Thanks @albertcthomas . I have extended your test to make it even more interesting. It's really nice that the POSIX semantics are such that the test pass out-of-the box on Linux and macOS.

Still we will have to rework the temporary file collection to have it work on windows as well. I have to think more about it but unfortunately I won't have time to work on it today. Maybe next week hopefully.

And the test fails for both loky and multiprocessing, showing that this error is indeed somewhat different to the one exposed in #942

I have extended your test to make it even more interesting. It's really nice that the POSIX semantics are such that the test pass out-of-the box on Linux and macOS.

And the views returned to the parent process are indeed memmaps.
Isn't it a bit "dangerous" to have a view being returned by default without the users being aware of it? (as memmapping is turned on by default for large data sets)

Isn't it a bit "dangerous" to have a view being returned by default without the users being aware of it? (as memmapping is turned on by default for large data sets)

Why? If you write the same code using the threading backend you would also have shared memory.

Why? If you write the same code using the threading backend you would also have shared memory.

True :).

The solution being implemented in #966 would make it such that a Parallel function calls that returns values that are views of tempory memmapped arrays automatically created by joblib from the input argument will always be materialized back as regular numpy arrays.

Fixed and released in 0.15.0.