With the great effort by @anaruse in #2090, I've seen encouraging performance boosts. Below is a list of possible improvements I can think of, either for offering extensive support or for enabling even more boost. I am interested in knowing what I've missed or misunderstood.

• Support complex numbers (Fix bug in CUB + **Native support** of complex numbers in CUB  #2538)
• Allow using CUDA streams: (Support stream in CUB #2555)

if users set up a context manager like this

with stream:
    arr.sum()
    # do other stuff

The non-default stream should be honored. All of the CUB functions introduced in #2090 support an optional stream argument. We just need to pick up the current stream pointer during setup and modify the wrappers.

• Change the CUB wrappers from def to cdef or cpdef: (UPDATE: see Discussion for possible enhancements of the new CUB support #2519 (comment))

currently they are all Python def functions. Could be be beneficial for performance. In particular, if we don't want to expose those wrappers to end users, cdef would be a nice choice.

• Support batch reduction for contiguous arrays (Refactor CUB to support an explicit axis argument; Fix alignments for Thrust's complex types #2562):

currently only a full reduction is supported, but if a reduction over the last axes of a contiguous array of shape, say, (X, Y, Z), is needed, this seems possible with a naive loop over the remaining axes. In other words, in this case we can use CUB to do arr.sum(axis=2) or arr.sum(axis=(1,2)), assuming arr is C contiguous. This resembles the current treatment of PlanNd in the FFT module.

• Document how to enable CUB support if built from source: Need to set CUB_PATH and CUB_DISABLED.
  -> could be avoided if the CUB source code is bundled (Build the cupy.cuda.cub module by default #2584)
• Support argmin and argmax (Add CUB support for argmax() and argmin() #2596 enables a global (no axis) search)
• Support half-precision floating points (Add fp16 support of CUB #2600)
• Support F-contiguous arrays (Support CUB reduction for F-contiguous arrays #2682)
• Support sparse matrix operation (CUB-based CSR sparse matrix vector multiply #2698)
• Honor the keepdims argument (keepdims should always preserve all dimensions in CUB-based reductions #2725)

Question: (from #2508 (comment)): is Jenkins configured to test CUB functionalities? UPDATE: No, see #2538 (comment).