Description

According to https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#hardware-precision-matrix, FP8 precision is supported for SM 8.9 or later. Confirmed to pass the tests locally on RTX 4060 Ti.

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified