Pinned Loading

1. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 1

2. vllm-project/speculators vllm-project/speculators Public

   A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

   Python 516 104

3. INT8BlockwiseGEMM INT8BlockwiseGEMM Public

   2-step online int8 quantization method for Ampere.

   Cuda

4. DotTopkFusedKernel DotTopkFusedKernel Public

   C++

5. BlockSparseVLLM BlockSparseVLLM Public

   Python

6. MultiModal-Eagle3 MultiModal-Eagle3 Public

   Python