01. Intro While deploying an INT8-quantized LLM with vLLM on NVIDIA A100 GPU, I ran into this error during startup — pytorch/ao#2376: RuntimeError: self.size(0) needs to be greater than 16, but g...

Recently, I suffered from Menhera (mental issue), caused by an uncontrollable situation. For a reminder, I would like to record “escape manual”. 01. Cure mental with physical distance first Escap...

블로그를 시작했습니다. 잘 부탁드립니다. Most posts will be technical write-ups of things I’ve read, implemented, or debugged.