Summary

Add quantization export support for trained models to reduce size and enable edge deployment.

Background

Per trueno-aprender-stdlib-core-language-spec.md Section 13.4 (Model Persistence):

• Implement quantization (Q8_0, Q4_0) export

Requirements

1. Quantization Formats

   • Q8_0: 8-bit quantization (4x size reduction)
   • Q4_0: 4-bit quantization (8x size reduction)
   • Compatible with GGUF/llama.cpp ecosystem

2. API

   impl Model {
       fn quantize(&self, format: QuantFormat) -> QuantizedModel;
       fn save_quantized(&self, path: &str, format: QuantFormat) -> Result<(), Error>;
   }
   
   enum QuantFormat {
       Q8_0,
       Q4_0,
       Q4_1,
       Q5_0,
   }

3. Quality Preservation

   • Accuracy degradation < 1% for Q8_0
   • Accuracy degradation < 5% for Q4_0
   • Calibration dataset support for optimal quantization

Acceptance Criteria

• Q8_0 quantization implemented
• Q4_0 quantization implemented
• Quantized models can be saved/loaded
• Accuracy tests show acceptable degradation
• GGUF export compatibility

Related

• Ruchy spec: docs/specifications/trueno-aprender-stdlib-core-language-spec.md
• Integration: ruchy::stdlib::aprender_bridge