[MRG] Backend hints and shared memory constraints#595
[MRG] Backend hints and shared memory constraints#595ogrisel wants to merge 12 commits intojoblib:masterfrom
Conversation
Codecov Report
@@ Coverage Diff @@
## master #595 +/- ##
==========================================
+ Coverage 94.9% 95.03% +0.12%
==========================================
Files 38 38
Lines 5005 5094 +89
==========================================
+ Hits 4750 4841 +91
+ Misses 255 253 -2
Continue to review full report at Codecov.
|
|
Actually, have To make this consistent I will also rewrite this to use the |
joblib/parallel.py
Outdated
| if require == 'sharedmem' and not supports_sharedmem: | ||
| # This backend does not match the shared memory constraint: | ||
| # fallback to the default thead-based backend. | ||
| backend = BACKENDS[DEFAULT_THREAD_BACKEND]() |
There was a problem hiding this comment.
Would it be possible to add a log message here, and maybe line 94, saying that the backend was changed?
That would address @stephen-hoover's comment in #537 (comment)
172c353 to
dca756b
Compare
…he active parallel backend
d75ca85 to
fbae101
Compare
|
@jcrist @stephen-hoover @TomAugspurger @mrocklin @GaelVaroquaux I think this is ready for review. |
4ca224a to
7ee44b9
Compare
|
I messed up with git. I reopened the PR as #602. |
This is an alternative implementation of #537. I reimplemented it from scratch because joblib had diverged a bit and I did not agree with the semantics of constraints violations in #537.
TODO:
RandomForestClassifieron dask-distributed cluster and check that fit run trees in parallel (done with this branch of sklearn: scikit-learn/scikit-learn@master...ogrisel:joblib-backend-hints);prefer='threads'andrequire='sharedmem'or keep the currently implemented boolean flags for hinting and hard constraints;