Motivation

Fixed dense model load weight issue (#232) .

Modifications

Cherry-pick PR sgl-project#21019 and Add BF16 qkv z b a fusion and PTPC quant config

Accuracy Tests

397B-A17B BF16：

397B-A17B PTPC FP8：

27B BF16

• TP1：

• TP 2：

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.