{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T21:38:21Z","timestamp":1772833101004,"version":"3.50.1"},"reference-count":68,"publisher":"Wiley","issue":"6","license":[{"start":{"date-parts":[[2024,9,2]],"date-time":"2024-09-02T00:00:00Z","timestamp":1725235200000},"content-version":"am","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2024,9,2]],"date-time":"2024-09-02T00:00:00Z","timestamp":1725235200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000015","name":"U.S. Department of Energy","doi-asserted-by":"publisher","award":["DE\u2010AC02\u201006CH11357"],"award-info":[{"award-number":["DE\u2010AC02\u201006CH11357"]}],"id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006227","name":"Lawrence Livermore National Laboratory","doi-asserted-by":"publisher","award":["DE\u2010AC52\u201007NA27344"],"award-info":[{"award-number":["DE\u2010AC52\u201007NA27344"]}],"id":[{"id":"10.13039\/100006227","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006227","name":"Lawrence Livermore National Laboratory","doi-asserted-by":"publisher","award":["LANL grant 531711"],"award-info":[{"award-number":["LANL grant 531711"]}],"id":[{"id":"10.13039\/100006227","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006602","name":"Air Force Research Laboratory","doi-asserted-by":"publisher","award":["FA8750\u201019\u20102\u20101000"],"award-info":[{"award-number":["FA8750\u201019\u20102\u20101000"]}],"id":[{"id":"10.13039\/100006602","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["wires.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["WIREs Data Min &amp;amp; Knowl"],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Automatic differentiation is a popular technique for computing derivatives of computer programs. While automatic differentiation has been successfully used in countless engineering, science, and machine learning applications, it can sometimes nevertheless produce surprising results. In this paper, we categorize problematic usages of automatic differentiation, and illustrate each category with examples such as chaos, time\u2010averages, discretizations, fixed\u2010point loops, lookup tables, linear solvers, and probabilistic programs, in the hope that readers may more easily avoid or detect such pitfalls. We also review debugging techniques and their effectiveness in these situations.<\/jats:p><jats:p>This article is categorized under:<jats:list list-type=\"simple\">\n<jats:list-item><jats:p>Technologies &gt; Machine Learning<\/jats:p><\/jats:list-item>\n<\/jats:list><\/jats:p>","DOI":"10.1002\/widm.1555","type":"journal-article","created":{"date-parts":[[2024,9,3]],"date-time":"2024-09-03T02:26:17Z","timestamp":1725330377000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["A taxonomy of automatic differentiation pitfalls"],"prefix":"10.1002","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3479-6361","authenticated-orcid":false,"given":"Jan","family":"H\u00fcckelheim","sequence":"first","affiliation":[{"name":"MCS Division Argonne National Laboratory  Lemont Illinois USA"}]},{"given":"Harshitha","family":"Menon","sequence":"additional","affiliation":[{"name":"CASC Lawrence Livermore National Laboratory  Livermore California USA"}]},{"given":"William","family":"Moses","sequence":"additional","affiliation":[{"name":"MIT CSAIL  Cambridge Massachusetts USA"}]},{"given":"Bruce","family":"Christianson","sequence":"additional","affiliation":[{"name":"School of Physics, Engineering &amp; Computer Science University of Hertfordshire  Hatfield UK"}]},{"given":"Paul","family":"Hovland","sequence":"additional","affiliation":[{"name":"MCS Division Argonne National Laboratory  Lemont Illinois USA"}]},{"given":"Laurent","family":"Hasco\u00ebt","sequence":"additional","affiliation":[{"name":"Inria Sophia\u2010Antipolis Team Ecuador  Valbonne France"}]}],"member":"311","published-online":{"date-parts":[[2024,9,2]]},"reference":[{"key":"e_1_2_11_2_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.102.011301"},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-020-00214-x"},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.2172\/971984"},{"key":"e_1_2_11_5_1","doi-asserted-by":"publisher","DOI":"10.2514\/2.7456"},{"key":"e_1_2_11_6_1","volume-title":"Automatic differentiation of programs with discrete randomness","author":"Arya G.","year":"2023"},{"key":"e_1_2_11_7_1","first-page":"690","article-title":"Deep equilibrium models","volume":"32","author":"Bai S.","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_11_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459775"},{"key":"e_1_2_11_9_1","first-page":"11","article-title":"Derivatives and control in the presence of shocks","volume":"09","author":"Bardos C.","year":"2002","journal-title":"Computational Fluid Dynamics Journal"},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611971538"},{"key":"e_1_2_11_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0377-0427(00)00422-2"},{"issue":"1","key":"e_1_2_11_12_1","first-page":"5595","article-title":"Automatic differentiation in machine learning: A survey","volume":"18","author":"Baydin A. G.","year":"2017","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_11_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/0377-0427(94)90293-3"},{"key":"e_1_2_11_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/0377-0427(94)90294-1"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/567806.567807"},{"key":"e_1_2_11_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2017.08.002"},{"key":"e_1_2_11_17_1","volume-title":"A mathematical model for automatic differentiation in machine learning","author":"Bolte J.","year":"2020"},{"key":"e_1_2_11_18_1","unstructured":"Bradbury J. Frostig R. Hawkins P. Johnson M. J. Leary C. Maclaurin D. Necula G. Paszke A. VanderPlas J. Wanderman\u2010Milne S. &Zhang Q.(2018).JAX: Composable transformations of python+NumPy programs.http:\/\/github.com\/google\/jax"},{"key":"e_1_2_11_19_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556789408805572"},{"key":"e_1_2_11_20_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556789808805697"},{"key":"e_1_2_11_21_1","doi-asserted-by":"publisher","DOI":"10.2514\/6.2001-821"},{"key":"e_1_2_11_22_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556780500151984"},{"key":"e_1_2_11_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/12.863031"},{"key":"e_1_2_11_24_1","doi-asserted-by":"publisher","DOI":"10.1137\/120873558"},{"key":"e_1_2_11_25_1","volume-title":"46th AIAA fluid dynamics conference American Institute of Aeronautics and Astronautics","author":"Fikl A.","year":"2016"},{"key":"e_1_2_11_26_1","first-page":"43","volume-title":"Automatic differentiation of algorithms: Theory, implementation, and application","author":"Fischer H.","year":"1991"},{"key":"e_1_2_11_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/103147.103151"},{"key":"e_1_2_11_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1334"},{"key":"e_1_2_11_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/293686.293695"},{"key":"e_1_2_11_30_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556789208805503"},{"key":"e_1_2_11_31_1","doi-asserted-by":"publisher","DOI":"10.1137\/080727464"},{"key":"e_1_2_11_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-68942-3_4"},{"key":"e_1_2_11_33_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0962492902000132"},{"key":"e_1_2_11_34_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556789308805549"},{"key":"e_1_2_11_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00607-011-0162-z"},{"key":"e_1_2_11_36_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898717761"},{"key":"e_1_2_11_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s002110000178"},{"key":"e_1_2_11_38_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556788.2016.1146269"},{"key":"e_1_2_11_39_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218488598000094"},{"key":"e_1_2_11_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11075-015-0067-6"},{"key":"e_1_2_11_41_1","unstructured":"H\u00fcckelheim J. C.(2017).Discrete adjoints on many cores algorithmic differentiation of accelerated fluid simulations(PhD thesis). Queen Mary University of London."},{"key":"e_1_2_11_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-45231-5_17"},{"key":"e_1_2_11_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2012.01.001"},{"key":"e_1_2_11_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3498710"},{"key":"e_1_2_11_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CGO.2004.1281665"},{"key":"e_1_2_11_46_1","volume-title":"On the correctness of automatic differentiation for neural networks with machine\u2010representable parameters","author":"Lee W.","year":"2023"},{"key":"e_1_2_11_47_1","first-page":"6719","volume-title":"Advances in neural information processing systems","author":"Lee W.","year":"2020"},{"key":"e_1_2_11_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3571198"},{"key":"e_1_2_11_49_1","first-page":"3082","volume-title":"International conference on machine learning","author":"Liao R.","year":"2018"},{"key":"e_1_2_11_50_1","first-page":"1540","volume-title":"International conference on artificial intelligence and statistics","author":"Lorraine J.","year":"2020"},{"key":"e_1_2_11_51_1","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1305"},{"key":"e_1_2_11_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2014.04.056"},{"key":"e_1_2_11_53_1","first-page":"12472","article-title":"Instead of rewriting foreign code for machine learning, automatically synthesize fast gradients","volume":"33","author":"Moses W.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_11_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3458817.3476165"},{"key":"e_1_2_11_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC41404.2022.00065"},{"key":"e_1_2_11_56_1","doi-asserted-by":"publisher","DOI":"10.2514\/6.2000-667"},{"key":"e_1_2_11_57_1","volume-title":"The art of differentiating computer programs: An introduction to algorithmic differentiation","author":"Naumann U.","year":"2012"},{"key":"e_1_2_11_58_1","first-page":"4065","volume-title":"Proceedings of the 35th international conference on machine learning, vol. 80 of proceedings of machine learning research","author":"Parmas P.","year":"2018"},{"key":"e_1_2_11_59_1","unstructured":"Pascanu R. Mikolov T. &Bengio Y.(2012).On the difficulty of training recurrent neural networks.arXiv.https:\/\/arxiv.org\/abs\/1211.5063"},{"key":"e_1_2_11_60_1","unstructured":"Paszke A. Gross S. Chintala S. Chanan G. Yang E. DeVito Z. Lin Z. Desmaison A. Antiga L. &Lerer A.Automatic differentiation in PyTorch 2017."},{"key":"e_1_2_11_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3571236"},{"key":"e_1_2_11_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3356900"},{"key":"e_1_2_11_63_1","doi-asserted-by":"publisher","DOI":"10.1137\/0721062"},{"key":"e_1_2_11_64_1","volume-title":"Implementation and measurements of an efficient fixed point adjoint","author":"Taftaf A.","year":"2015"},{"key":"e_1_2_11_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEC.1959.5222693"},{"key":"e_1_2_11_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341700"},{"key":"e_1_2_11_67_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2012.09.007"},{"key":"e_1_2_11_68_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2014.03.002"},{"key":"e_1_2_11_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/356012.356013"}],"container-title":["WIREs Data Mining and Knowledge Discovery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/wires.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/widm.1555","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/wires.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/widm.1555","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,11]],"date-time":"2025-08-11T09:36:40Z","timestamp":1754905000000},"score":1,"resource":{"primary":{"URL":"https:\/\/wires.onlinelibrary.wiley.com\/doi\/10.1002\/widm.1555"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,2]]},"references-count":68,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["10.1002\/widm.1555"],"URL":"https:\/\/doi.org\/10.1002\/widm.1555","archive":["Portico"],"relation":{},"ISSN":["1942-4787","1942-4795"],"issn-type":[{"value":"1942-4787","type":"print"},{"value":"1942-4795","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,2]]},"assertion":[{"value":"2023-05-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e1555"}}