{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T22:12:29Z","timestamp":1770243149233,"version":"3.49.0"},"reference-count":40,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA2","license":[{"start":{"date-parts":[[2024,10,8]],"date-time":"2024-10-08T00:00:00Z","timestamp":1728345600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["389792660"],"award-info":[{"award-number":["389792660"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2024,10,8]]},"abstract":"<jats:p>Bugs in popular distributed protocol implementations have been the source of many downtimes in popular internet services. We describe a randomized testing approach for distributed protocol implementations based on reinforcement learning. Since the natural reward structure is very sparse, the key to successful exploration in reinforcement learning is reward augmentation. We show two different techniques that build on one another. First, we provide a decaying exploration bonus based on the discovery of new states\u2014the reward decays as the same state is visited multiple times. The exploration bonus captures the intuition from coverage-guided fuzzing of prioritizing new coverage points; in contrast to other schemes, we show that taking the maximum of the bonus and the Q-value leads to more effective exploration. Second, we provide waypoints to the algorithm as a sequence of predicates that capture interesting semantic scenarios. Waypoints exploit designer insight about the protocol and guide the exploration to \u201cinteresting\u201d parts of the state space. Our reward structure ensures that new episodes can reliably get to deep interesting states even without execution caching. We have implemented our algorithm in Go. Our evaluation on three large benchmarks (RedisRaft, Etcd, and RSL) shows that our algorithm can significantly outperform baseline approaches in terms of coverage and bug finding.<\/jats:p>","DOI":"10.1145\/3689779","type":"journal-article","created":{"date-parts":[[2024,10,8]],"date-time":"2024-10-08T03:23:04Z","timestamp":1728357784000},"page":"1928-1954","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Reward Augmentation in Reinforcement Learning for Testing Distributed Systems"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-4842-6599","authenticated-orcid":false,"given":"Andrea","family":"Borgarelli","sequence":"first","affiliation":[{"name":"MPI-SWS, Kaiserslautern, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2727-8865","authenticated-orcid":false,"given":"Constantin","family":"Enea","sequence":"additional","affiliation":[{"name":"LIX, CNRS, \u00c9cole polytechnique, Institut Polytechnique de Paris, Palaiseau, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2136-0542","authenticated-orcid":false,"given":"Rupak","family":"Majumdar","sequence":"additional","affiliation":[{"name":"MPI-SWS, Kaiserslautern, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7171-5543","authenticated-orcid":false,"given":"Srinidhi","family":"Nagendra","sequence":"additional","affiliation":[{"name":"IRIF - CNRS - Universit\u00e9 Paris Cit\u00e9, Paris, France"},{"name":"Chennai Mathematical Institute, Chennai, India"}]}],"member":"320","published-online":{"date-parts":[[2024,10,8]]},"reference":[{"key":"e_1_3_1_2_1","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1007\/978-3-031-22337-2_29","volume-title":"Principles of Systems Design - Essays Dedicated to Thomas A. Henzinger on the Occasion of His 60th Birthday (Lecture Notes in Computer Science, Vol. 13660)","author":"Alur Rajeev","year":"2022","unstructured":"Rajeev Alur, Suguman Bansal, Osbert Bastani, and Kishor Jothimurugan. 2022. A Framework for Transforming Specifications in Reinforcement Learning. In Principles of Systems Design - Essays Dedicated to Thomas A. Henzinger on the Occasion of His 60th Birthday (Lecture Notes in Computer Science, Vol. 13660), Jean-Fran\u00e7ois Raskin, Krishnendu Chatterjee, Laurent Doyen, and Rupak Majumdar (Eds.). Springer, 604\u2013624. https:\/\/doi.org\/10.1007\/978-3-031-22337-2_29 10.1007\/978-3-031-22337-2_29"},{"key":"e_1_3_1_3_1","unstructured":"Andrea Borgarelli and Srinidhi Nagendra. 2024. Artifact for reward augmentation in reinforcement learning for testing distributed systems. https:\/\/doi.org\/10.5281\/zenodo.13166254 10.5281\/zenodo.13166254"},{"key":"e_1_3_1_4_1","first-page":"116","volume-title":"2018 IEEE Security and Privacy Workshops, SP Workshops 2018, San Francisco, CA, USA, May 24, 2018","author":"Bottinger Konstantin","year":"2018","unstructured":"Konstantin Bottinger, Patrice Godefroid, and Rishabh Singh. 2018. Deep Reinforcement Fuzzing. In 2018 IEEE Security and Privacy Workshops, SP Workshops 2018, San Francisco, CA, USA, May 24, 2018. IEEE Computer Society, 116\u2013122. https:\/\/doi.org\/10.1109\/SPW.2018.00026 10.1109\/SPW.2018.00026"},{"key":"e_1_3_1_5_1","first-page":"167","volume-title":"Proceedings of the 15th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2010, Pittsburgh, Pennsylvania, USA, March 13-17, 2010","author":"Burckhardt Sebastian","year":"2010","unstructured":"Sebastian Burckhardt, Pravesh Kothari, Madanlal Musuvathi, and Santosh Nagarakatte. 2010. A randomized scheduler with probabilistic guarantees of finding bugs. In Proceedings of the 15th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2010, Pittsburgh, Pennsylvania, USA, March 13-17, 2010, James C. Hoe and Vikram S. Adve (Eds.). ACM, 167\u2013178. https:\/\/doi.org\/10.1145\/1736020.1736040 10.1145\/1736020.1736040"},{"key":"e_1_3_1_6_1","first-page":"423","volume-title":"15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14-16, 2021","author":"Chajed Tej","year":"2021","unstructured":"Tej Chajed, Joseph Tassarotti, Mark Theng, Ralf Jung, M. Frans Kaashoek, and Nickolai Zeldovich. 2021. GoJournal: a verified, concurrent, crash-safe journaling system. In 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14-16, 2021, Angela Demke Brown and Jay R. Lorch (Eds.). USENIX Association, 423\u2013439. https:\/\/www.usenix.org\/conference\/osdi21\/presentation\/chajed"},{"key":"e_1_3_1_7_1","first-page":"2325","volume-title":"29th USENIX Security Symposium, USENIX Security 2020, August 12-14, 2020","author":"Chen Hongxu","year":"2020","unstructured":"Hongxu Chen, Shengjian Guo, Yinxing Xue, Yulei Sui, Cen Zhang, Yuekang Li, Haijun Wang, and Yang Liu. 2020. MUZZ: Thread-aware Grey-box Fuzzing for Effective Bug Hunting in Multithreaded Programs. In 29th USENIX Security Symposium, USENIX Security 2020, August 12-14, 2020, Srdjan Capkun and Franziska Roesner (Eds.). USENIX Association, 2325\u20132342. https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/chen-hongxu"},{"key":"e_1_3_1_8_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.639"},{"key":"e_1_3_1_9_1","first-page":"210:1","article-title":"Testing consensus implementations using communication closure","volume":"4","author":"Dragoi Cezara","year":"2020","unstructured":"Cezara Dragoi, Constantin Enea, Burcu Kulahcioglu Ozkan, Rupak Majumdar, and Filip Niksic. 2020. Testing consensus implementations using communication closure. Proc. ACM Program. Lang. 4, OOPSLA (2020), 210:1\u2013210:29. https:\/\/doi.org\/10.1145\/3428278 10.1145\/3428278","journal-title":"Proc. ACM Program. Lang."},{"key":"e_1_3_1_10_1","first-page":"328","volume-title":"Proceedings of the Twelfth European Conference on Computer Systems, EuroSys 2017, Belgrade, Serbia, April 23-26, 2017","author":"Fonseca Pedro","year":"2017","unstructured":"Pedro Fonseca, Kaiyuan Zhang, Xi Wang, and Arvind Krishnamurthy. 2017. An Empirical Study on the Correctness of Formally Verified Distributed Systems. In Proceedings of the Twelfth European Conference on Computer Systems, EuroSys 2017, Belgrade, Serbia, April 23-26, 2017, Gustavo Alonso, Ricardo Bianchini, and Marko Vukolic (Eds.). ACM, 328\u2013343. https:\/\/doi.org\/10.1145\/3064176.3064183 10.1145\/3064176.3064183"},{"key":"e_1_3_1_11_1","first-page":"2211","volume-title":"45th IEEE\/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20, 2023","author":"Gao Yu","year":"2023","unstructured":"Yu Gao, Wensheng Dou, Dong Wang, Wenhan Feng, Jun Wei, Hua Zhong, and Tao Huang. 2023. Coverage Guided Fault Injection for Cloud Systems. In 45th IEEE\/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20, 2023. IEEE, 2211\u20132223. https:\/\/doi.org\/10.1109\/ICSE48619.2023.00186 10.1109\/ICSE48619.2023.00186"},{"key":"e_1_3_1_12_1","first-page":"1","volume-title":"Proceedings of the 25th Symposium on Operating Systems Principles, SOSP 2015, Monterey, CA, USA, October 4-7, 2015","author":"Hawblitzel Chris","year":"2015","unstructured":"Chris Hawblitzel, Jon Howell, Manos Kapritsos, Jacob R. Lorch, Bryan Parno, Michael Lowell Roberts, Srinath T. V. Setty, and Brian Zill. 2015. IronFleet: proving practical distributed systems correct. In Proceedings of the 25th Symposium on Operating Systems Principles, SOSP 2015, Monterey, CA, USA, October 4-7, 2015, Ethan L. Miller and Steven Hand (Eds.). ACM, 1\u201317. https:\/\/doi.org\/10.1145\/2815400.2815428 10.1145\/2815400.2815428"},{"key":"e_1_3_1_13_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.12440"},{"key":"e_1_3_1_14_1","first-page":"7995","volume-title":"Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021","author":"Jiang Yuqian","year":"2021","unstructured":"Yuqian Jiang, Suda Bharadwaj, Bo Wu, Rishi Shah, Ufuk Topcu, and Peter Stone. 2021. Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Tasks. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. AAAI Press, 7995\u20138003. https:\/\/doi.org\/10.1609\/AAAI.V35I9.16975 10.1609\/AAAI.V35I9.16975"},{"key":"e_1_3_1_15_1","first-page":"4870","volume-title":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119)","author":"Jin Chi","year":"2020","unstructured":"Chi Jin, Akshay Krishnamurthy, Max Simchowitz, and Tiancheng Yu. 2020. Reward-Free Exploration for Reinforcement Learning. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119). PMLR, 4870\u20134879. http:\/\/proceedings.mlr.press\/v119\/jin20d.html"},{"key":"e_1_3_1_16_1","unstructured":"Kishor Jothimurugan Suguman Bansal Osbert Bastani and Rajeev Alur. 2021. Compositional Reinforcement Learning from Logical Specifications. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021 NeurIPS 2021 December 6-14 2021 virtual Marc\u2019Aurelio Ranzato Alina Beygelzimer Yann N. Dauphin Percy Liang and Jennifer Wortman Vaughan (Eds.). 10026\u201310039. https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/531db99cb00833bcd414459069dc7387-Abstract.html"},{"key":"e_1_3_1_17_1","unstructured":"Kyle Kingsbury. 2020. Jepsen. https:\/\/jepsen.io"},{"key":"e_1_3_1_18_1","first-page":"399","volume-title":"11th USENIX Symposium on Operating Systems Design and Implementation, OSDI \u201814, Broomfield, CO, USA, October 6-8, 2014","author":"Leesatapornwongsa Tanakorn","year":"2014","unstructured":"Tanakorn Leesatapornwongsa, Mingzhe Hao, Pallavi Joshi, Jeffrey F. Lukman, and Haryadi S. Gunawi. 2014. SAMC: Semantic-Aware Model Checking for Fast Discovery of Deep Bugs in Cloud Systems. In 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI \u201814, Broomfield, CO, USA, October 6-8, 2014, Jason Flinn and Hank Levy (Eds.). USENIX Association, 399\u2013414. https:\/\/www.usenix.org\/conference\/osdi14\/technical-sessions\/presentation\/leesatapornwongsa"},{"key":"e_1_3_1_19_1","first-page":"1615","volume-title":"Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS 2023, Copenhagen, Denmark, November 26-30, 2023","author":"Meng Ruijie","year":"2023","unstructured":"Ruijie Meng, George Pirlea, Abhik Roychoudhury, and Ilya Sergey. 2023. Greybox Fuzzing of Distributed Systems. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS 2023, Copenhagen, Denmark, November 26-30, 2023, Weizhi Meng, Christian Damsgaard Jensen, Cas Cremers, and Engin Kirda (Eds.). ACM, 1615\u20131629. https:\/\/doi.org\/10.1145\/3576915.3623097 10.1145\/3576915.3623097"},{"key":"e_1_3_1_20_1","first-page":"230:1","article-title":"Learning-based controlled concurrency testing","volume":"4","author":"Mukherjee Suvam","year":"2020","unstructured":"Suvam Mukherjee, Pantazis Deligiannis, Arpita Biswas, and Akash Lal. 2020. Learning-based controlled concurrency testing. Proc. ACM Program. Lang. 4, OOPSLA (2020), 230:1\u2013230:31. https:\/\/doi.org\/10.1145\/3428298 10.1145\/3428298","journal-title":"Proc. ACM Program. Lang."},{"key":"e_1_3_1_21_1","first-page":"160:1","article-title":"Randomized testing of distributed systems with probabilistic guarantees","volume":"2","author":"Ozkan Burcu Kulahcioglu","year":"2018","unstructured":"Burcu Kulahcioglu Ozkan, Rupak Majumdar, Filip Niksic, Mitra Tabaei Befrouei, and Georg Weissenbacher. 2018. Randomized testing of distributed systems with probabilistic guarantees. Proc. ACM Program. Lang. 2, OOPSLA (2018), 160:1\u2013160:28. https:\/\/doi.org\/10.1145\/3276530 10.1145\/3276530","journal-title":"Proc. ACM Program. Lang."},{"key":"e_1_3_1_22_1","first-page":"180:1","article-title":"Trace aware random testing for distributed systems","volume":"3","author":"Ozkan Burcu Kulahcioglu","year":"2019","unstructured":"Burcu Kulahcioglu Ozkan, Rupak Majumdar, and Simin Oraee. 2019. Trace aware random testing for distributed systems. Proc. ACM Program. Lang. 3, OOPSLA (2019), 180:1\u2013180:29. https:\/\/doi.org\/10.1145\/3360606 10.1145\/3360606","journal-title":"Proc. ACM Program. Lang."},{"key":"e_1_3_1_23_1","first-page":"1043","volume-title":"Advances in Neural Information Processing Systems 10, [NIPS Conference, Denver, Colorado, USA, 1997]","author":"Parr Ronald","year":"1997","unstructured":"Ronald Parr and Stuart Russell. 1997. Reinforcement Learning with Hierarchies of Machines. In Advances in Neural Information Processing Systems 10, [NIPS Conference, Denver, Colorado, USA, 1997], Michael I. Jordan, Michael J. Kearns, and Sara A. Solla (Eds.). The MIT Press, 1043\u20131049. http:\/\/papers.nips.cc\/paper\/1384-reinforcement-learning-with-hierarchies-of-machines"},{"key":"e_1_3_1_24_1","first-page":"1410","volume-title":"ICSE \u201820: 42nd International Conference on Software Engineering, Seoul, South Korea, 27 June - 19 July, 2020","author":"Reddy Sameer","year":"2020","unstructured":"Sameer Reddy, Caroline Lemieux, Rohan Padhye, and Koushik Sen. 2020. Quickly generating diverse valid test inputs with reinforcement learning. In ICSE \u201820: 42nd International Conference on Software Engineering, Seoul, South Korea, 27 June - 19 July, 2020, Gregg Rothermel and Doo-Hwan Bae (Eds.). ACM, 1410\u20131421. https:\/\/doi.org\/10.1145\/3377811.3380399 10.1145\/3377811.3380399"},{"key":"e_1_3_1_25_1","first-page":"11","volume-title":"Proceedings of the ACM SIGPLAN 2008 Conference on Programming Language Design and Implementation, Tucson, AZ, USA, June 7-13, 2008","author":"Sen Koushik","year":"2008","unstructured":"Koushik Sen. 2008. Race directed random testing of concurrent programs. In Proceedings of the ACM SIGPLAN 2008 Conference on Programming Language Design and Implementation, Tucson, AZ, USA, June 7-13, 2008, Rajiv Gupta and Saman P. Amarasinghe (Eds.). ACM, 11\u201321. https:\/\/doi.org\/10.1145\/1375581.1375584 10.1145\/1375581.1375584"},{"key":"e_1_3_1_26_1","first-page":"188","volume-title":"Model Checking Software - 18th International SPIN Workshop, Snowbird, UT, USA, July 14-15, 2011. Proceedings (Lecture Notes in Computer Science, Vol. 6823)","author":"Simsa Jiri","year":"2011","unstructured":"Jiri Simsa, Randy Bryant, and Garth A. Gibson. 2011. dBug: Systematic Testing of Unmodified Distributed and Multithreaded Systems. In Model Checking Software - 18th International SPIN Workshop, Snowbird, UT, USA, July 14-15, 2011. Proceedings (Lecture Notes in Computer Science, Vol. 6823), Alex Groce and Madanlal Musuvathi (Eds.). Springer, 188\u2013193. https:\/\/doi.org\/10.1007\/978-3-642-22306-8_14 10.1007\/978-3-642-22306-8_14"},{"key":"e_1_3_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.1998.712192"},{"key":"e_1_3_1_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(99)00052-1"},{"key":"e_1_3_1_29_1","first-page":"36","volume-title":"45th IEEE\/ACM International Conference on Software Engineering: Software Engineering in Practice, SEIP@ICSE 2023, Melbourne, Australia, May 14-20, 2023","author":"Meerten Martijn","year":"2023","unstructured":"Martijn van Meerten, Burcu Kulahcioglu Ozkan, and Annibale Panichella. 2023. Evolutionary Approach for Concurrency Testing of Ripple Blockchain Consensus Algorithm. In 45th IEEE\/ACM International Conference on Software Engineering: Software Engineering in Practice, SEIP@ICSE 2023, Melbourne, Australia, May 14-20, 2023. IEEE, 36\u201347. https:\/\/doi.org\/10.1109\/ICSE-SEIP58684.2023.00009 10.1109\/ICSE-SEIP58684.2023.00009"},{"key":"e_1_3_1_30_1","first-page":"240","volume-title":"Formal Approaches to Software Testing and Runtime Verification, First Combined International Workshops, FATES 2006 and RV 2006, Seattle, WA, USA, August 15-16, 2006, Revised Selected Papers (Lecture Notes in Computer Science, Vol. 4262)","author":"Veanes Margus","year":"2006","unstructured":"Margus Veanes, Pritam Roy, and Colin Campbell. 2006. Online Testing with Reinforcement Learning. In Formal Approaches to Software Testing and Runtime Verification, First Combined International Workshops, FATES 2006 and RV 2006, Seattle, WA, USA, August 15-16, 2006, Revised Selected Papers (Lecture Notes in Computer Science, Vol. 4262), Klaus Havelund, Manuel Nunez, Grigore Rosu, and Burkhart Wolff (Eds.). Springer, 240\u2013253. https:\/\/doi.org\/10.1007\/11940197_16 10.1007\/11940197_16"},{"key":"e_1_3_1_31_1","first-page":"221","volume-title":"Proceedings of the 33rd International Conference on Software Engineering, ICSE 2011, Waikiki, Honolulu, HI, USA, May 21-28, 2011","author":"Wang Chao","year":"2011","unstructured":"Chao Wang, Mahmoud Said, and Aarti Gupta. 2011. Coverage guided systematic concurrency testing. In Proceedings of the 33rd International Conference on Software Engineering, ICSE 2011, Waikiki, Honolulu, HI, USA, May 21-28, 2011, Richard N. Taylor, Harald C. Gall, and Nenad Medvidovic (Eds.). ACM, 221\u2013230. https:\/\/doi.org\/10.1145\/1985793.1985824 10.1145\/1985793.1985824"},{"key":"e_1_3_1_32_1","first-page":"127","volume-title":"Proceedings of the Eighteenth European Conference on Computer Systems, EuroSys 2023, Rome, Italy, May 8-12, 2023","author":"Wang Dong","year":"2023","unstructured":"Dong Wang, Wensheng Dou, Yu Gao, Chenao Wu, Jun Wei, and Tao Huang. 2023. Model Checking Guided Testing for Distributed Systems. In Proceedings of the Eighteenth European Conference on Computer Systems, EuroSys 2023, Rome, Italy, May 8-12, 2023, Giuseppe Antonio Di Luna, Leonardo Querzoni, Alexandra Fedorova, and Dushyanth Narayanan (Eds.). ACM, 127\u2013143. https:\/\/doi.org\/10.1145\/3552326.3587442 10.1145\/3552326.3587442"},{"key":"e_1_3_1_33_1","first-page":"2741","volume-title":"30th USENIX Security Symposium, USENIX Security 2021, August 11-13, 2021","author":"Wang Daimeng","year":"2021","unstructured":"Daimeng Wang, Zheng Zhang, Hang Zhang, Zhiyun Qian, Srikanth V. Krishnamurthy, and Nael B. Abu-Ghazaleh. 2021. SyzVegas: Beating Kernel Fuzzing Odds with Reinforcement Learning. In 30th USENIX Security Symposium, USENIX Security 2021, August 11-13, 2021, Michael D. Bailey and Rachel Greenstadt (Eds.). USENIX Association, 2741\u20132758. https:\/\/www.usenix.org\/conference\/usenixsecurity21\/presentation\/wang-daimeng"},{"key":"e_1_3_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"key":"e_1_3_1_35_1","first-page":"357","volume-title":"Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Portland, OR, USA, June 15-17, 2015","author":"Wilcox James R.","year":"2015","unstructured":"James R. Wilcox, Doug Woos, Pavel Panchekha, Zachary Tatlock, Xi Wang, Michael D. Ernst, and Thomas E. Anderson. 2015. Verdi: a framework for implementing and formally verifying distributed systems. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Portland, OR, USA, June 15-17, 2015, David Grove and Stephen M. Blackburn (Eds.). ACM, 357\u2013368. https:\/\/doi.org\/10.1145\/2737924.2737958 10.1145\/2737924.2737958"},{"key":"e_1_3_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586053"},{"key":"e_1_3_1_37_1","first-page":"4010","volume-title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019","author":"Xu Zhe","year":"2019","unstructured":"Zhe Xu and Ufuk Topcu. 2019. Transfer of Temporal Logic Formulas in Reinforcement Learning. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, Sarit Kraus (Ed.). ijcai.org, 4010\u20134018. https:\/\/doi.org\/10.24963\/IJCAI.2019\/557 10.24963\/IJCAI.2019\/557"},{"key":"e_1_3_1_38_1","first-page":"213","volume-title":"Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2009, April 22-24, 2009, Boston, MA, USA","author":"Yang Junfeng","year":"2009","unstructured":"Junfeng Yang, Tisheng Chen, Ming Wu, Zhilei Xu, Xuezheng Liu, Haoxiang Lin, Mao Yang, Fan Long, Lintao Zhang, and Lidong Zhou. 2009. MODIST: Transparent Model Checking of Unmodified Distributed Systems. In Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2009, April 22-24, 2009, Boston, MA, USA, Jennifer Rexford and Emin G\u00fcn Sirer (Eds.). USENIX Association, 213\u2013228. http:\/\/www.usenix.org\/events\/nsdi09\/tech\/full_papers\/yang\/yang.pdf"},{"key":"e_1_3_1_39_1","first-page":"317","volume-title":"Computer Aided Verification - 30th International Conference, CAV 2018, Held as Part of the Federated Logic Conference, FloC 2018, Oxford, UK, July 14-17, 2018, Proceedings, Part II (Lecture Notes in Computer Science, Vol. 10982)","author":"Yuan Xinhao","year":"2018","unstructured":"Xinhao Yuan, Junfeng Yang, and Ronghui Gu. 2018. Partial Order Aware Concurrency Sampling. In Computer Aided Verification - 30th International Conference, CAV 2018, Held as Part of the Federated Logic Conference, FloC 2018, Oxford, UK, July 14-17, 2018, Proceedings, Part II (Lecture Notes in Computer Science, Vol. 10982), Hana Chockler and Georg Weissenbacher (Eds.). Springer, 317\u2013335. https:\/\/doi.org\/10.1007\/978-3-319-96142-2_20 10.1007\/978-3-319-96142-2_20"},{"key":"e_1_3_1_40_1","first-page":"555","volume-title":"36th IEEE\/ACM International Conference on Automated Software Engineering, ASE 2021, Melbourne, Australia, November 15-19, 2021","author":"Zhang Shaohua","year":"2021","unstructured":"Shaohua Zhang, Shuang Liu, Jun Sun, Yuqi Chen, Wenzhi Huang, Jinyi Liu, Jian Liu, and Jianye Hao. 2021. FIGCPS: Effective Failure-inducing Input Generation for Cyber-Physical Systems with Deep Reinforcement Learning. In 36th IEEE\/ACM International Conference on Automated Software Engineering, ASE 2021, Melbourne, Australia, November 15-19, 2021. IEEE, 555\u2013567. https:\/\/doi.org\/10.1109\/ASE51524.2021.9678832 10.1109\/ASE51524.2021.9678832"},{"key":"e_1_3_1_41_1","unstructured":"Xuezhou Zhang Yuzhe Ma and Adish Singla. 2020. Task-agnostic Exploration in Reinforcement Learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 NeurIPS 2020 December 6-12 2020 virtual Hugo Larochelle Marc\u2019Aurelio Ranzato Raia Hadsell Maria-Florina Balcan and Hsuan-Tien Lin (Eds.). https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/8763d72bba4a7ade23f9ae1f09f4efc7-Abstract.html"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3689779","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3689779","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T09:05:58Z","timestamp":1770195958000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3689779"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,8]]},"references-count":40,"journal-issue":{"issue":"OOPSLA2","published-print":{"date-parts":[[2024,10,8]]}},"alternative-id":["10.1145\/3689779"],"URL":"https:\/\/doi.org\/10.1145\/3689779","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,8]]},"assertion":[{"value":"2024-04-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-18","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}