problems with CUDA 12.1: missing named symbol #201
Description
I compiled bitsandbytes from source for tloen/alpaca-lora and CUDA_VERSION=121, but execution failed with this error:
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 121
CUDA SETUP: Loading binary <REDACTED>/alpaca-lora/venv/lib/python3.10/site-packages/bitsandbytes-0.37.1-py3.10.egg/bitsandbytes/libbitsandbytes_cuda121.so...
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
Error named symbol not found at line 508 in file <REDACTED>/alpaca-lora/bitsandbytes/csrc/ops.cu
What else can I do to figure out what went wrong? is CUDA 12.1 supported yet?
The system is a Ubuntu 22.04 with a RTX 3060 (driver 530). Other pytorch setups work without issue.