* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once

* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once

* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once

* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once

support fp8 dispatch

* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* set a higher timeout threshold to prevent forced terminated

* disable rope kernel to address the accuracy regression in llama

 add pd disaggregation best practices

Signed-off-by: Stanley Winata <stanley.winata@amd.com>

[Wave] Add wave extend attention kernel

Signed-off-by: Harsh Menon <harsh@nod-labs.com>

[Wave] Adding logit_cap and layer scaling to API

Also add support for the wave backend to the model
runner. And use Triton decode kernels for now.

[Wave] Run chunked prefill for perf comparison on Wave test

Need to rename the non chunked/regular prefill version because otherwise
rpd will treat it as the same kernel

Signed-off-by: Stanley Winata <stanley.winata@amd.com>

[Wave] Cache the function that loads the wave kernel

Also maintain a global kernel hash to avoid
recomputing the hash on every call.

[Wave] Don't specify block size and enable buffer ops

[Wave] Enable wave runtime and update scheduling API

[Wave] Update API to use wave_compile & WaveCompileOptions

[Wave] Update wave backend and extend attention to latest

[Wave] Add speculative decode kernel

Signed-off-by: nithinsubbiah <nithinsubbiah@gmail.com>

cache kernels using lru_cache

Update WaveBackend to use Wave Decode  (sgl-project#6)

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

Revert "Update WaveBackend to use Wave Decode  (sgl-project#6)" (sgl-project#7)

This reverts commit eac4599.

Wave Backend decode (sgl-project#8)

* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

Wave backend fixes (sgl-project#10)

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

More fixes to Wave decode (sgl-project#12)

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

is_causal

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

Enable the grok in3 model (sgl-project#14)

Set unique cache dir for each worker (sgl-project#16)

update kernel (sgl-project#18)

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

updated spec decode test as per wave

Signed-off-by: xintin <gaurav.verma@amd.com>

fix extend (sgl-project#23)

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

Refactor paged decode intermediate arrays shapes (sgl-project#24)

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

remove dyn symbols (sgl-project#26)

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

cleanup shapes (sgl-project#27)

Some fields were removed from `paged_decode_attention_shape`.

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

Remove `mha` param from Wave decode attention kernel (sgl-project#28)

Depends on iree-org/iree-turbine#1039

Signed-off-by: Paul Zhang <paul.zhang@amd.com>

nfc: fix problems reported by linting

update references from iree.turbine to wave_lang

# This is the 1st commit message:

rebase

# This is the commit message sgl-project#2:

remove duplicated code

# This is the commit message sgl-project#3:

add type hints

# This is the commit message sgl-project#4:

add clear cache for benchmark alignment

# This is the commit message sgl-project#5:

remove unuse arg

# This is the commit message sgl-project#6:

clear cache once

# This is the commit message sgl-project#7:

simplified VAE cache logic for qwenimage and wan

# This is the commit message sgl-project#8:

remove duplicated code

Deepseek V3.2 support

[FEAT] Support dense&sparse together

* [npu]adaptation to deterministic inference

* modify review comments

Log Session Id Patch

31B dense model with bidirectional attention fix