Currently there're 4 CUB-related environment variables.

• CUB_PATH: used at build time to build cupy.cuda.cub
• CUPY_CUB_PATH: used at runtime to build reduction kernel (Performance boost: CUB-backed _SimpleReductionKernel #3244)
• CUPY_CUB_BLOCK_REDUCTION_DISABLED (default: 1): Set to 0 to enable performance boost in reduction kernel (Performance boost: CUB-backed _SimpleReductionKernel #3244)
• CUB_DISABLED (default: 1): Set to 0 to enable CUB in various computations

We think it is better to disable CUB by default, even in CUDA 11, as users expect reproducibility (deterministic behavior). Instead, we need to provide a simple "opt-in" method to enable CUB for users needing a performance boost.

This issue proposes the followings:

• Remove CUB_PATH and CUPY_CUB_PATH (always use bundled CUB Build the cupy.cuda.cub module by default #2584 or CUDA 11 built-CUB, as user-provided CUB headers are unlikely to be used (considering we haven't requested to implement CUPY_THRUST_PATH).
  • In CUDA 11, use CUB included in CUDA runtime. If CUB headers are unavailable in system (e.g., maybe in "runtime" docker container; need to check after CUDA 11 is finally released), use CUB bundled in CuPy.
  • In CUDA 10 or earlier, always use CUB bundled in CuPy.
• Merge CUPY_CUB_BLOCK_REDUCTION_DISABLED and CUB_DISABLED into one environment variable CUPY_BACKENDS="cub,cutensor" (default: "", i.e., cub/cutensor disabled by default). Users can specify backends in the referred order, separated by a comma. Boolean flags (cupy.core.cub_block_reduction_enabled and cupy.cuda.cub_enabled) are removed for consistency.
• For testing and debugging purposes, provide the following undocumented APIs:

cupy.core.set_routine_backends(['cub', 'cutensor'])
cupy.core.set_reduction_backends(['cub', 'cutensor'])

(edit 2020-06-25: updated based on the latest discussion)

cc/ @leofang @anaruse