There is a common gotcha in Cython code: even after a type check Cython compiles txt.encode('utf-8') to a code which looks up 'encode' method, then calls it as a Python method, this method has to find utf-8 codec, etc.

if isinstance(f, unicode):
    f = f.encode("utf-8")

On the other hand,

if isinstance(f, unicode):
    f = (<unicode>f).encode("utf-8")

compiles directly to a PyUnicode_AsUTF8String call.

Another gotcha is that if you declare argument type (e.g. bytes) and then pass a variable of type object, it does more work, not less work because type is checked. Because we know a variable is bytes casting it to bytes allow to remove all these type checks and conversions.

The third thing I fixed is string_types lookup. "basestring" in Cython is the same as six.string_types; isinstance check gets compiled directly to C API calls this way.

These changes allow to make HashingVectorizer about 10% faster. Benchmark script:

from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction.text import HashingVectorizer

categories = ['alt.atheism', 'comp.graphics']
remove = ('headers', 'footers', 'quotes')
data_train = fetch_20newsgroups(subset='train', categories=categories,
                                shuffle=True, random_state=42,
                                remove=remove)
vec = HashingVectorizer()
%timeit vec.fit_transform(data_train.data[:100])

For me it runs in 13.8 ms before the changes and 11.5 ms after the changes.

This change is