Unless I'm missing something, it's not completely trivial how one can use a custom sklearn.tree._criterion.Criterion for a decision tree. See my use case here.

Things I have tried include:

• Import the ClassificationCriterion in Python and subclass it. It seems that node_impurity and children_impurity do not get called, the impurity is always 0 (perhaps because they are cdef and not cpdef?). I'm also unsure what the parameters to __new__ / __cinit__ should be (e.g. 1 and np.array([2], dtype='intp') for a binary classification problem?), or how to pass them properly: I have to create the Criterion object from outside the tree to circumvent the check on the criterion argument.

• Extend ClassificationCriterion in a Cython file. This seems to work, but (a) it requires exporting ClassificationCriterion from _criterion.pxd and (b) it would be nice if it would be documented more extensively what should be done in node_impurity and children_impurity. I will post my code below once it seems to work correctly.

May I propose one of the following to make this easier?

• Document what should be done to extend the class in Cython or Python - if Python should be allowed: I am aware of the performance issue with that, but in some cases it may be OK to do this in Python - I don't know.
• Make it possible to pass a function or other object not extending Criterion to the tree, similar to how it is very easy to implement a custom scorer for validation functions. That would require changing the checks here.