{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T06:44:15Z","timestamp":1777099455378,"version":"3.51.4"},"reference-count":39,"publisher":"Wiley","issue":"12","license":[{"start":{"date-parts":[[2018,9,11]],"date-time":"2018-09-11T00:00:00Z","timestamp":1536624000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"publisher","award":["406308\/2016-0"],"award-info":[{"award-number":["406308\/2016-0"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"publisher","award":["453611\/2017-6"],"award-info":[{"award-number":["453611\/2017-6"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"publisher","award":["304220\/2017-5"],"award-info":[{"award-number":["304220\/2017-5"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006162","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Ci\u00eancia e Tecnologia do Estado de Pernambuco","doi-asserted-by":"publisher","award":["APQ-0839-1.03\/14"],"award-info":[{"award-number":["APQ-0839-1.03\/14"]}],"id":[{"id":"10.13039\/501100006162","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006162","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Ci\u00eancia e Tecnologia do Estado de Pernambuco","doi-asserted-by":"publisher","award":["APQ 0388-1.03\/14"],"award-info":[{"award-number":["APQ 0388-1.03\/14"]}],"id":[{"id":"10.13039\/501100006162","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Softw Pract Exp"],"published-print":{"date-parts":[[2018,12]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>Continuous integration (CI) is a software development practice that has been gaining increasing popularity in the last few years. However, we still miss a collection of experiences regarding how software developers perceive the idea of CI, in terms of its fundamental concepts, the reasons that motivate the adoption of this practice, the reasons for build breakage, and the benefits and problems related to CI. To shed light on this direction, we conducted a user survey with 158 CI users. Through a mostly qualitative investigation, we produce a list of findings that are not always obvious. For instance, we observed that (1) developers are not sure whether a job failure represents a failure or not; (2) inadequate testing is the most common technical reason related to build breakage, whereas lack of time plays a role on the social reasons; and (3) although some respondents reported that CI systems increase the confidence that the code is in a known state, some respondents also reported that there is a false sense of confidence when blindly trusting tests. This empirical study is particularly relevant to those interested in better understanding and fostering CI practices either in an open\u2010source or industrial setting.<\/jats:p>","DOI":"10.1002\/spe.2637","type":"journal-article","created":{"date-parts":[[2018,9,11]],"date-time":"2018-09-11T17:04:44Z","timestamp":1536685484000},"page":"2223-2236","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Work practices and challenges in continuous integration: A survey with Travis CI users"],"prefix":"10.1002","volume":"48","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7598-2799","authenticated-orcid":false,"given":"Gustavo","family":"Pinto","sequence":"first","affiliation":[{"name":"Federal University of Par\u00e1  66075\u2010110 Bel\u00e9m\u2010PA Brazil"}]},{"given":"Fernando","family":"Castor","sequence":"additional","affiliation":[{"name":"Federal University of Pernambuco  50740\u2010560 Recife\u2010PE Brazil"}]},{"given":"Rodrigo","family":"Bonifacio","sequence":"additional","affiliation":[{"name":"University of Bras\u00edlia  73345\u2010010 Bras\u00edlia\u2010DF Brazil"}]},{"given":"Marcel","family":"Rebou\u00e7as","sequence":"additional","affiliation":[{"name":"In Loco Media  50030\u2010310 Recife\u2010PE Brazil"}]}],"member":"311","published-online":{"date-parts":[[2018,9,11]]},"reference":[{"key":"e_1_2_14_2_1","volume-title":"Extreme Programming Explained: Embrace Change","author":"Beck K","year":"2004"},{"key":"e_1_2_14_3_1","volume-title":"Continuous Integration: Improving Software Quality and Reducing Risk","author":"Duvall P","year":"2007"},{"key":"e_1_2_14_4_1","doi-asserted-by":"crossref","unstructured":"HiltonM NelsonN TunnellT MarinovD DigD.Trade\u2010offs in continuous integration: assurance security and flexibility. In: Proceedings of the 2017 11Th Joint Meeting on Foundations of Software Engineering (ESEC\/FSE);2017;Paderborn Germany.","DOI":"10.1145\/3106237.3106270"},{"key":"e_1_2_14_5_1","doi-asserted-by":"crossref","unstructured":"BellerM GousiosG ZaidmanA.Oops my tests broke the build: an explorative analysis of Travis CI with GitHub. In: Proceedings of the 2017 IEEE\/ACM 14th International Conference on Mining Software Repositories (MSR);2017;Buenos Aires Argentina.","DOI":"10.1109\/MSR.2017.62"},{"key":"e_1_2_14_6_1","doi-asserted-by":"crossref","unstructured":"HiltonM TunnellT HuangK MarinovD DigD.Usage costs and benefits of continuous integration in open\u2010source projects. In: Proceedings of the 31st IEEE\/ACM International Conference on Automated Software Engineering (ASE);2016;Singapore Singapore.","DOI":"10.1145\/2970276.2970358"},{"key":"e_1_2_14_7_1","unstructured":"NicolaiJ.Github welcomes all CI tools.2000.http:\/\/web.archive.org\/web\/20180626090647\/https:\/\/blog.github.com\/2017-11-07-github-welcomes-all-ci-tools\/"},{"key":"e_1_2_14_8_1","doi-asserted-by":"crossref","unstructured":"BellerM.Toward an empirical theory of feedback\u2010driven development. In: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings (ICSE);2018;Gothenburg Sweden.","DOI":"10.1145\/3183440.3190332"},{"key":"e_1_2_14_9_1","unstructured":"FowlerM.Continuous integration (original version).2000.http:\/\/web.archive.org\/web\/20180626090647\/https:\/\/www.martinfowler.com\/articles\/originalContinuousIntegration.html"},{"key":"e_1_2_14_10_1","doi-asserted-by":"crossref","unstructured":"BellerM GousiosG PanichellaA ZaidmanA.When how and why developers (do not) test in their IDEs. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC\/FSE);2015;Bergamo Italy.","DOI":"10.1145\/2786805.2786843"},{"key":"e_1_2_14_11_1","doi-asserted-by":"crossref","unstructured":"LuoQ HaririF EloussiL MarinovD.An empirical analysis of flaky tests. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE);2014;Hong Kong China.","DOI":"10.1145\/2635868.2635920"},{"key":"e_1_2_14_12_1","doi-asserted-by":"crossref","unstructured":"BellerM GousiosG ZaidmanA.How (much) do developers test?In: Proceedings of the 37th International Conference on Software Engineering (ICSE);2015;Florence Italy.","DOI":"10.1109\/ICSE.2015.193"},{"key":"e_1_2_14_13_1","doi-asserted-by":"crossref","unstructured":"GousiosG StoreyMA BacchelliA.Work practices and challenges in pull\u2010based development: the contributor's perspective. In: Proceedings of the IEEE\/ACM 38th International Conference on Software Engineering (ICSE);2016;Austin TX.","DOI":"10.1145\/2884781.2884826"},{"key":"e_1_2_14_14_1","doi-asserted-by":"crossref","unstructured":"GousiosG ZaidmanA StoreyMD vanDeursenA.Work practices and challenges in pull\u2010based development: the integrator's perspective. In: Proceedings of the 37Th IEEE\/ACM International Conference on Software Engineering (ICSE);2015;Florence Italy.","DOI":"10.1109\/ICSE.2015.55"},{"key":"e_1_2_14_15_1","doi-asserted-by":"crossref","unstructured":"PintoG Rebou\u00e7asM CastorF.Inadequate testing time pressure and (over) confidence: a tale of continuous integration users. In: Proceedings of the 10th IEEE\/ACM International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE);2017;Buenos Aires Argentina.","DOI":"10.1109\/CHASE.2017.13"},{"key":"e_1_2_14_16_1","doi-asserted-by":"crossref","unstructured":"RayB PosnettD FilkovV DevanbuP.A large scale study of programming languages and code quality in GitHub. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE);2014;Hong Kong China.","DOI":"10.1145\/2635868.2635922"},{"key":"e_1_2_14_17_1","doi-asserted-by":"crossref","unstructured":"BorgesH HoraA ValenteMT.Understanding the factors that impact the popularity of GitHub repositories. Paper presented at: 32nd IEEE International Conference on Software Maintenance and Evolution (ICSME);2016;Raleigh NC.","DOI":"10.1109\/ICSME.2016.31"},{"key":"e_1_2_14_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1027796"},{"key":"e_1_2_14_19_1","doi-asserted-by":"crossref","unstructured":"SmithE LoftinR Murphy\u2010HillE BirdC ZimmermannT.Improving developer participation rates in surveys. Paper presented at: 2013 6th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE);2013;San Francisco CA.https:\/\/doi.org\/10.1109\/CHASE.2013.6614738","DOI":"10.1109\/CHASE.2013.6614738"},{"key":"e_1_2_14_20_1","volume-title":"Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory","author":"Strauss A","year":"2007"},{"key":"e_1_2_14_21_1","doi-asserted-by":"crossref","unstructured":"VasilescuB YuY WangH DevanbuP FilkovV.Quality and productivity outcomes relating to continuous integration in GitHub. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC\/FSE);2015;Bergamo Italy.","DOI":"10.1145\/2786805.2786850"},{"key":"e_1_2_14_22_1","unstructured":"FowlerM.Continuous integration: integrate at least daily.2018.http:\/\/web.archive.org\/web\/20180626090647\/https:\/\/www.thoughtworks.com\/continuous-integration"},{"key":"e_1_2_14_23_1","doi-asserted-by":"crossref","unstructured":"ZhaoY SerebrenikA ZhouY FilkovV VasilescuB.The impact of continuous integration on other software development practices: a large\u2010scale empirical study. In: Proceedings of the 32nd IEEE\/ACM International Conference on Automated Software Engineering (ASE);2017;Urbana IL.","DOI":"10.1109\/ASE.2017.8115619"},{"key":"e_1_2_14_24_1","doi-asserted-by":"crossref","unstructured":"TrockmanA ZhouS K\u00e4stnerC VasilescuB.Adding sparkle to social coding: an empirical study of repository badges in thenpmecosystem. In: Proceedings of the 40th International Conference on Software Engineering (ICSE);2018;Gothenburg Sweden.","DOI":"10.1145\/3183440.3190335"},{"key":"e_1_2_14_25_1","doi-asserted-by":"crossref","unstructured":"VasilescuB vanSchuylenburgS WulmsJ SerebrenikA van denBrandMGJ.Continuous integration in a social\u2010coding world: empirical evidence from GitHub. Paper presented at: 30th IEEE International Conference on Software Maintenance and Evolution;2014;Victoria Canada.","DOI":"10.1109\/ICSME.2014.62"},{"key":"e_1_2_14_26_1","doi-asserted-by":"crossref","unstructured":"LuzW PintoG Bonif\u00e1cioR.Building a collaborative culture: a grounded theory of well succeeded devOps adoption in practice. Paper presented at: 2018 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM);2018;Oulu Finland.","DOI":"10.1145\/3239235.3240299"},{"key":"e_1_2_14_27_1","doi-asserted-by":"crossref","unstructured":"SeoH SadowskiC ElbaumS AftandilianE BowdidgeR.Programmers' build errors: a case study (at Google). In: Proceedings of the 36th International Conference on Software Engineering (ICSE);2014;Hyderabad India.","DOI":"10.1145\/2568225.2568255"},{"key":"e_1_2_14_28_1","doi-asserted-by":"crossref","unstructured":"SteinmacherI WieseIS ConteT GerosaMA RedmilesDF.The hard life of open source software project newcomers. In: Proceedings of the 7th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE);2014;Hyderabad India.","DOI":"10.1145\/2593702.2593704"},{"key":"e_1_2_14_29_1","doi-asserted-by":"crossref","unstructured":"PintoG SteinmacherI GerosaMA.More common than you think: an in\u2010depth study of casual contributors. Paper presented at: 2016 IEEE 23rd International Conference on Software Analysis Evolution and Reengineering (SANER);2016;Suita Japan.","DOI":"10.1109\/SANER.2016.68"},{"key":"e_1_2_14_30_1","doi-asserted-by":"crossref","unstructured":"T\u00f3masd\u00f3ttirKF AnicheMF vanDeursenA.Why and how JavaScript developers use linters. In: Proceedings of the 32nd IEEE\/ACM International Conference on Automated Software Engineering (ASE);2017;Urbana IL.","DOI":"10.1109\/ASE.2017.8115668"},{"key":"e_1_2_14_31_1","first-page":"1","volume-title":"Structured Programming","author":"Dijkstra EW","year":"1972"},{"key":"e_1_2_14_32_1","doi-asserted-by":"crossref","unstructured":"WilliamsTW MercerMR MuchaJP KapurR.Code coverage what does it mean in terms of quality?Paper presented at: Annual Reliability and Maintainability Symposium. 2001 Proceedings. International Symposium on Product Quality and Integrity;2001;Philadelphia PA.https:\/\/doi.org\/10.1109\/RAMS.2001.902502","DOI":"10.1109\/RAMS.2001.902502"},{"key":"e_1_2_14_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/267580.267590"},{"key":"e_1_2_14_34_1","doi-asserted-by":"crossref","unstructured":"BriandL PfahlD.Using simulation for assessing the real impact of test coverage on defect coverage. In: Proceedings of the 10th International Symposium on Software Reliability Engineering (ISSRE);1999;Boca Raton FL.","DOI":"10.1109\/ISSRE.1999.809319"},{"key":"e_1_2_14_35_1","doi-asserted-by":"crossref","unstructured":"AbramoviciM ParikhPS.WARNING: 100% fault coverage may be misleading!!In: Proceedings of the IEEE International Test Conference;1992;Baltimore MD.https:\/\/doi.org\/10.1109\/TEST.1992.527887","DOI":"10.1109\/TEST.1992.527887"},{"key":"e_1_2_14_36_1","doi-asserted-by":"crossref","unstructured":"AdamsB TrompH de\u00a0SchutterK de\u00a0MeuterW.Design recovery and maintenance of build systems. Paper presented at: 2007 IEEE International Conference on Software Maintenance;2007;Paris France.https:\/\/doi.org\/10.1109\/ICSM.2007.4362624","DOI":"10.1109\/ICSM.2007.4362624"},{"key":"e_1_2_14_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-011-9169-5"},{"key":"e_1_2_14_38_1","doi-asserted-by":"crossref","unstructured":"BellerM GousiosG ZaidmanA.TravisTorrent: synthesizing Travis CI and GitHub for full\u2010stack research on continuous integration. In: Proceedings of the 14th International Conference on Mining Software Repositories (MSR);2017;Buenos Aires Argentina.","DOI":"10.1109\/MSR.2017.24"},{"key":"e_1_2_14_39_1","doi-asserted-by":"crossref","unstructured":"Rebou\u00e7asM SantosRO PintoG CastorF.How does contributors' involvement influence the build status of an open\u2010source software project?In: Proceedings of the 14th International Conference on Mining Software Repositories (MSR);2017;Buenos Aires Argentina.","DOI":"10.1109\/MSR.2017.32"},{"key":"e_1_2_14_40_1","doi-asserted-by":"crossref","unstructured":"KimS ParkS YunJ LeeY.Automated continuous integration of component\u2010based software: an industrial experience. In: Proceedings of the 2008 23rd IEEE\/ACM International Conference on Automated Software Engineering (ASE);2008;L'Aquila Italy.","DOI":"10.1109\/ASE.2008.64"}],"container-title":["Software: Practice and Experience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fspe.2637","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/spe.2637","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,17]],"date-time":"2023-09-17T06:35:31Z","timestamp":1694932531000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/spe.2637"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,11]]},"references-count":39,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2018,12]]}},"alternative-id":["10.1002\/spe.2637"],"URL":"https:\/\/doi.org\/10.1002\/spe.2637","archive":["Portico"],"relation":{},"ISSN":["0038-0644","1097-024X"],"issn-type":[{"value":"0038-0644","type":"print"},{"value":"1097-024X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,9,11]]},"assertion":[{"value":"2018-03-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-08-08","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-09-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}