[Bug]: fix inconsistent round-trip mapping for FP8 variants#2467
[Bug]: fix inconsistent round-trip mapping for FP8 variants#2467Shaoting-Feng merged 17 commits intoLMCache:devfrom
Conversation
Summary of ChangesHello @hlin99, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request resolves a critical bug in the string mapping of PyTorch's FP8 floating-point data types. By providing unique string identifiers for Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
The pull request effectively resolves a critical issue where torch.float8_e4m3fn and torch.float8_e4m3fnuz were mapped to the same string identifier. By assigning unique string representations to these FP8 variants, the change ensures precise and idempotent dtype serialization, which is crucial for maintaining hardware compatibility and preventing precision errors during reverse lookup operations. This is a significant correctness improvement.
Prior to this change, both 'float8_e4m3fn' and 'float8_e4m3fnuz' were mapped to the same string 'fp8_e4m3'. This caused an issue where 'float8_e4m3fn' would be incorrectly restored as 'float8_e4m3fnuz' during reverse lookup, leading to potential hardware incompatibility and precision errors. This commit assigns unique string identifiers to each FP8 variant: - torch.float8_e4m3fn -> "fp8_e4m3fn" - torch.float8_e4m3fnuz -> "fp8_e4m3fnuz" This ensures precise and idempotent dtype serialization. Signed-off-by: Tony Lin <tony.lin@intel.com>
|
hi @kobe0938 @maobaolong would you like to have a review? without this fix, fp8_e4m3fn & fp8_e5m2 won't work properly due to this N->1 & 1->N mapping are not tracable. |
|
hi @YaoJiayi @Shaoting-Feng the patch is to fix a bug where dtype -> str -> dtype conversion was not reversible. I saw you are the authors of relevant code. can you help review and suggest? thank a lot. |
an example is: |
maobaolong
left a comment
There was a problem hiding this comment.
this lgtm, @sammshen @Shaoting-Feng Would you do another look?
sammshen
left a comment
There was a problem hiding this comment.
LGTM, thanks for the fix!
…2467) fix(utils): fix inconsistent round-trip mapping for FP8 variants Prior to this change, both 'float8_e4m3fn' and 'float8_e4m3fnuz' were mapped to the same string 'fp8_e4m3'. This caused an issue where 'float8_e4m3fn' would be incorrectly restored as 'float8_e4m3fnuz' during reverse lookup, leading to potential hardware incompatibility and precision errors. This commit assigns unique string identifiers to each FP8 variant: - torch.float8_e4m3fn -> "fp8_e4m3fn" - torch.float8_e4m3fnuz -> "fp8_e4m3fnuz" This ensures precise and idempotent dtype serialization. Signed-off-by: Tony Lin <tony.lin@intel.com>
…2467) fix(utils): fix inconsistent round-trip mapping for FP8 variants Prior to this change, both 'float8_e4m3fn' and 'float8_e4m3fnuz' were mapped to the same string 'fp8_e4m3'. This caused an issue where 'float8_e4m3fn' would be incorrectly restored as 'float8_e4m3fnuz' during reverse lookup, leading to potential hardware incompatibility and precision errors. This commit assigns unique string identifiers to each FP8 variant: - torch.float8_e4m3fn -> "fp8_e4m3fn" - torch.float8_e4m3fnuz -> "fp8_e4m3fnuz" This ensures precise and idempotent dtype serialization. Signed-off-by: Tony Lin <tony.lin@intel.com>
…2467) fix(utils): fix inconsistent round-trip mapping for FP8 variants Prior to this change, both 'float8_e4m3fn' and 'float8_e4m3fnuz' were mapped to the same string 'fp8_e4m3'. This caused an issue where 'float8_e4m3fn' would be incorrectly restored as 'float8_e4m3fnuz' during reverse lookup, leading to potential hardware incompatibility and precision errors. This commit assigns unique string identifiers to each FP8 variant: - torch.float8_e4m3fn -> "fp8_e4m3fn" - torch.float8_e4m3fnuz -> "fp8_e4m3fnuz" This ensures precise and idempotent dtype serialization. Signed-off-by: Tony Lin <tony.lin@intel.com>
…2467) fix(utils): fix inconsistent round-trip mapping for FP8 variants Prior to this change, both 'float8_e4m3fn' and 'float8_e4m3fnuz' were mapped to the same string 'fp8_e4m3'. This caused an issue where 'float8_e4m3fn' would be incorrectly restored as 'float8_e4m3fnuz' during reverse lookup, leading to potential hardware incompatibility and precision errors. This commit assigns unique string identifiers to each FP8 variant: - torch.float8_e4m3fn -> "fp8_e4m3fn" - torch.float8_e4m3fnuz -> "fp8_e4m3fnuz" This ensures precise and idempotent dtype serialization. Signed-off-by: Tony Lin <tony.lin@intel.com>
…2467) fix(utils): fix inconsistent round-trip mapping for FP8 variants Prior to this change, both 'float8_e4m3fn' and 'float8_e4m3fnuz' were mapped to the same string 'fp8_e4m3'. This caused an issue where 'float8_e4m3fn' would be incorrectly restored as 'float8_e4m3fnuz' during reverse lookup, leading to potential hardware incompatibility and precision errors. This commit assigns unique string identifiers to each FP8 variant: - torch.float8_e4m3fn -> "fp8_e4m3fn" - torch.float8_e4m3fnuz -> "fp8_e4m3fnuz" This ensures precise and idempotent dtype serialization. Signed-off-by: Tony Lin <tony.lin@intel.com> Signed-off-by: shaoxiawjc <wjc2800@163.com>
Prior to this change, both 'float8_e4m3fn' and 'float8_e4m3fnuz' were mapped to the same string 'fp8_e4m3'. This caused an issue where 'float8_e4m3fn' would be incorrectly restored as 'float8_e4m3fnuz' during reverse lookup, leading to potential hardware incompatibility and precision errors.
This commit assigns unique string identifiers to each FP8 variant:
This ensures precise and idempotent dtype serialization.