Skip to content

Possible regression in nested parallelism #12389

@glemaitre

Description

@glemaitre

I think that we have a possible regression linked to the nested parallelism. Let's take the following minimum example:

import numpy as np                                                                   
from sklearn.ensemble import RandomForestClassifier                                  
from sklearn.model_selection import cross_val_score                                  
                                                                                     
X = np.random.randn(500000, 80)                                                      
y = np.random.randint(0, 2, 500000)                                                  
                                                                                     
clf = RandomForestClassifier(n_jobs=-1)                                                               
cross_val_score(clf, X, y, n_jobs=1) 

In 0.19.2, htop shows activity for all processors. In 0.20, a single core is active. Is it link to the heuristic to avoid to over-subscription?

Can anyone try to reproduce it? ping @tomMoral @ogrisel @pierreglaser

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions