James Bagrow and Josh Bongard

Kolmogorov-Arnold Networks (KANs) uniquely combine high accuracy with interpretability, making them valuable for scientific modeling. However, it is unclear a priori how deep a network needs to be for any given task, and deeper KANs can be difficult to optimize and interpret. Here we introduce multi-exit KANs, where each layer includes its own prediction branch, enabling the network to make accurate predictions at multiple depths simultaneously. This architecture provides deep supervision that improves training while discovering the right level of model complexity for each task. Multi-exit KANs consistently outperform standard, single-exit versions on synthetic functions, dynamical systems, and real-world datasets. Remarkably, the best predictions often come from earlier, simpler exits, revealing that these networks naturally identify smaller, more parsimonious and interpretable models without sacrificing accuracy. To automate this discovery, we develop a differentiable "learning-to-exit" algorithm that balances contributions from exits during training. Our approach offers scientists a practical way to achieve both high performance and interpretability, addressing a fundamental challenge in machine learning for scientific discovery.

PyKAN and its dependencies.

https://github.com/KindXiaoming/pykan

Please see the included notebook.

For more information, and to cite our work, please see our paper:

Multi-exit Kolmogorov–Arnold networks: enhancing accuracy and parsimony
James Bagrow and Josh Bongard 2025 Mach. Learn.: Sci. Technol. 6 035037

• Journal link: https://dx.doi.org/10.1088/2632-2153/adf9bd
• arXiv link: https://arxiv.org/abs/2506.03302

@article{Bagrow_2025,
    doi = {10.1088/2632-2153/adf9bd},
    url = {https://dx.doi.org/10.1088/2632-2153/adf9bd},
    year = {2025},
    month = {aug},
    publisher = {IOP Publishing},
    volume = {6},
    number = {3},
    pages = {035037},
    author = {Bagrow, James and Bongard, Josh},
    title = {Multi-exit Kolmogorov–Arnold networks: enhancing accuracy and parsimony},
    journal = {Machine Learning: Science and Technology},
}

TY  - JOUR
DO  - 10.1088/2632-2153/adf9bd
UR  - https://dx.doi.org/10.1088/2632-2153/adf9bd
TI  - Multi-exit Kolmogorov–Arnold networks: enhancing accuracy and parsimony
T2  - Machine Learning: Science and Technology
AU  - Bagrow, James
AU  - Bongard, Josh
PY  - 2025
DA  - 2025/08/21
PB  - IOP Publishing
SP  - 035037
IS  - 3
VL  - 6
SN  - 2632-2153
SN  - 
ER  -