from torchao.quantization import quantize_, Int8WeightOnlyConfig
quantize_(model, Int8WeightOnlyConfig())

this scheme would be nice to have for Gemma models (and others, too, i assume)

i might see about getting around to this one