This repository reproduces the method from the paper:

Embarrassingly Simple Self-Distillation Improves Code Generation

The approach consists of three simple steps:

1. Sample solutions from a frozen model at non-unit temperature
2. Fine-tune on raw, unverified outputs using standard cross-entropy
3. Decode with a separately tuned temperature

No rewards · No verifier · No teacher · No RL

For full details, see the paper.

• [2026-04-03] 🚀 Initial release of repository
• [2026-04-03] 🤗 Model checkpoints coming soon on Hugging Face
• (More updates will be added here)

git clone https://github.com/apple/ml-ssd.git
cd ml-ssd
uv sync --group evaluation

source .venv/bin/activate
python evaluation/eval.py \
    --model <hf_model_name> \
    --tensor_parallel_size 4 \
    --max_tokens 65536 \
    --n_repeat 10 \
    --sampling_params "temperature=0.9,top_p=0.8,top_k=20" \
    --output_path ./results/

Note: Model checkpoints are coming soon. Stay tuned!

├── evaluation/
│   ├── eval.py                  # CLI entry point
│   ├── benchmark.py             # LiveCodeBench v6 implementation
│   └── livecodebench_utils.py   # Code execution utilities
├── figures/
│   └── fig_teaser.png
├── pyproject.toml
└── README.md

@misc{zhang2026embarrassinglysimpleselfdistillationimproves,
      title={Embarrassingly Simple Self-Distillation Improves Code Generation},
      author={Ruixiang Zhang and Richard He Bai and Huangjie Zheng and Navdeep Jaitly and Ronan Collobert and Yizhe Zhang},
      year={2026},
      eprint={2604.01193},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2604.01193},
}