@jiqing-feng @Titus-von-Koeller Please help review these changes. These changes add support for NF4 quantization/dequantization using Intel Gaudi hardware. https://www.intel.com/content/www/us/en/products/details/processors/ai-accelerators/gaudi.html
This PR adds support for only single level NF4 quantization for now, we are working on adding support for second level NF4 quantization and will add that using another PR in near future. Thanks!

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

The code looks good, thanks for your work on this, a promising first step!

@vivekgoe @ckvermaAI

Please see this short update about the multi-backend refactor #1596.

Regarding the Intel backend, as discussed in parallel with Ke Ding, the target for PRs migrating existing work from multi-backend-refactor instead of main will be the new bitsandbytes-intel repo.

However, some of the pure torch ops and generic cpu functionality still make more sense in the main branch of bitsandbytes, if they don't have the Intel IPEX dependency. Please align with @matthewdouglas and me on those. It's probably best to discuss that in our shared Slack channel.

@Titus-von-Koeller Thanks for reviewing and merging our PR! If possible, please add me to shared Slack channel you mentioned or if it needs to be done by someone in Intel team then let me know I will follow-up internally.