Enabled int8 weights by default

…i_docker

Docker.ubi: add missing package git

Within the existing `decoding` request parameter section:
  
  ```protobuf
  enum ResponseFormat {
    // Plain text, no constraints
    TEXT = 0;
    // Valid json
    JSON = 1;
  }

  message StringChoices {
    repeated string choices = 1;
  }

  // Mutually-exclusive guided decoding options
  oneof guided {
    // Output will be in the specified format
    ResponseFormat format = 3;
    // Output will follow the provided JSON schema
    string json_schema = 4;
    // Output will follow the provided regex pattern
    string regex = 5;
    // Output will be exactly one of the specified choices
    StringChoices choice = 6;
    // Output will follow the provided context free grammar
    string grammar = 7;
  }
```

Signed-off-by: Nick Hill <nickhill@us.ibm.com>

…on_opts

vLLM lm head optimization (tpp)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Co-authored-by: dengyunyang <dengyunyang@huawei.com>

…oject#26)

* indexer medatata to separate prefill and decode

* deep_gemm prefill kernel

* decode kernel, can run for single batch

* bug fixing insert decode k into kv before gemm

* don't use tilelang quant function

* faster non-looping torch for kv cache insertion

* add chunked prefill impl

* change quant kernel back to tilelang for promotion

* fix format (vllm-project#31)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* update unit tests

* Fp8 indexer prefill (vllm-project#33)

* init

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* can run

---------

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* remove debug comment

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* cleanup

* further cleanup

---------

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>

…t in TTModelRunner::_make_sampler_output as expected by vLLM downstream

…-rebase

DBO+aclgrph

…reset on `apt-get` (vllm-project#30784)" (vllm-project#31)

This reverts commit 2a60ac9.

* [Docker][Dev] Fix libnccl-dev version for the CUDA 13.0.1 devel image

[Docker][Dev] Fix libnccl-dev version conflict for the CUDA 13.0.1 devel image

Further update

* feat: Support FA4 for mm-encoder-attn-backend for qwen models

* feat: Kernel warmup for vit fa4

* fix: Fix some minor conflicts due to the introduction of flash_attn.cute

* Revert "[Docker][Dev] Fix libnccl-dev version for the CUDA 13.0.1 devel image"

This reverts commit ab76b28.

* chore: Update requirements and revert README.md

* chore: Install git for flash_attn cute installation

* lint: Fix linting

* Revert "[Improvement] Persist CUDA compat libraries paths to prevent reset on `apt-get` (vllm-project#30784)" (vllm-project#31)

This reverts commit 2a60ac9.

---------

Co-authored-by: Shang Wang <shangw@nvidia.com>