As mentioned in this paper - TEQ: Trainable Equivalent Transformation for Quantization of LLMs.
The authors of this paper are claiming - "The training process is lightweight, requiring only 1K steps and less than 1‰ of the original model’s trainable parameters."
Is this in the pipeline? It would be great if unsloth can support this.
https://arxiv.org/pdf/2310.10944.pdf
https://github.com/intel/neural-compressor
Thank you for building this awesome library!
As mentioned in this paper - TEQ: Trainable Equivalent Transformation for Quantization of LLMs.
The authors of this paper are claiming - "The training process is lightweight, requiring only 1K steps and less than 1‰ of the original model’s trainable parameters."
Is this in the pipeline? It would be great if unsloth can support this.
https://arxiv.org/pdf/2310.10944.pdf
https://github.com/intel/neural-compressor
Thank you for building this awesome library!