Description

• Improved the efficiency of manual cast with type checking.
• Apply the behavior of precision=full into manual cast
  • Also ensure fp8 can be used with precision=full on autocastable devices.
    (With manual cast, which is as fast as autocast + output upcast. I just chose the simples one)
• Use float32 for MHA layer in ipex since some computation in it is not good for IPEX devices.
• Have checked this implementation can be used on NV Cuda/Apple M/Intel ARC. With latest ipex, Arc A770 can use manual cast with fp8 features.
• Manual Cast on A770 is now faster then autocast('xpu'), even with ipex.optimize.

Checklist:

• I have read contributing wiki page
• I have performed a self-review of my own code
• My code follows the style guidelines
• My code passes tests