{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T07:47:34Z","timestamp":1761896854448},"reference-count":35,"publisher":"IGI Global","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,7]]},"abstract":"<jats:p>Glowworm swarm optimization (GSO) is one of the optimization techniques, which need to be parallelized in order to evaluate large problems with high-dimensional function spaces. There are various issues involved in the parallelization of any algorithm such as efficient communication among nodes in a cluster, load balancing, automatic node failure recovery, and scalability of nodes at runtime. In this article, the authors have implemented the GSO algorithm with the Apache Spark framework. Even though we need to address how to distribute the data in the cluster to improve the efficiency of algorithm, the Spark framework is designed in such a way that one does not need to deal with any actual underlying parallelization details. For the experimentation, two multimodal benchmark functions were used to evaluate the Spark-GSO algorithm with various sizes of dimensionality. The authors evaluate the optimization results of the two evaluation functions as well as they will compare the Spark results with the ones obtained using a previously implemented MapReduce-based GSO algorithm.<\/jats:p>","DOI":"10.4018\/ijsir.2018070101","type":"journal-article","created":{"date-parts":[[2018,7,18]],"date-time":"2018-07-18T15:34:07Z","timestamp":1531928047000},"page":"1-22","source":"Crossref","is-referenced-by-count":6,"title":["Comparing Spark with MapReduce"],"prefix":"10.4018","volume":"9","author":[{"given":"Goutham","family":"Miryala","sequence":"first","affiliation":[{"name":"North Dakota State University, Fargo, USA"}]},{"given":"Simone A.","family":"Ludwig","sequence":"additional","affiliation":[{"name":"Department of Computer Science, North Dakota State University, Fargo, USA"}]}],"member":"2432","reference":[{"key":"IJSIR.2018070101-0","doi-asserted-by":"publisher","DOI":"10.1109\/SIS.2013.6615155"},{"key":"IJSIR.2018070101-1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2013.6557888"},{"key":"IJSIR.2018070101-2","doi-asserted-by":"publisher","DOI":"10.4018\/IJSIR.2016010102"},{"key":"IJSIR.2018070101-3","unstructured":"Apache Spark. (2017) Apache Spark - Lightning-Fast Cluster Computing. Retrieved November 17, 2017, from https:\/\/spark.apache.org\/"},{"key":"IJSIR.2018070101-4","unstructured":"Apache Spark @Scale. (2017). Apache Spark @Scale: A 60 TB+ production use case from Facebook - The Databricks Blog. Retrieved November 17, 2017 from https:\/\/databricks.com\/blog\/2016\/08\/31\/apache-spark-scale-a-60-tb-production-use-case.html"},{"key":"IJSIR.2018070101-5","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04225-6_2"},{"key":"IJSIR.2018070101-6","unstructured":"Dean, J., & Ghemawat, S. (2004). MapReduce: simplified data processing on large clusters. In Proceedings of the 6th Conference on Symposium on Opearting Systems Design & Implementation \u2013 (Vol. 6). USENIX Association. Retrieved from http:\/\/dl.acm.org\/citation.cfm?id=1251264"},{"key":"IJSIR.2018070101-7","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-58484-6_252"},{"key":"IJSIR.2018070101-8","doi-asserted-by":"publisher","DOI":"10.1002\/9780470512517"},{"issue":"1","key":"IJSIR.2018070101-9","first-page":"975","article-title":"Comparing Apache Spark and Map Reduce with Performance Analysis using K-Means.","volume":"113","author":"S.Gopalani","year":"2015","journal-title":"International Journal of Computers and Applications"},{"key":"IJSIR.2018070101-10","author":"A.Grama","year":"2003","journal-title":"Introduction to parallel computing"},{"key":"IJSIR.2018070101-11","doi-asserted-by":"publisher","DOI":"10.1109\/ETECH.2004.1353848"},{"key":"IJSIR.2018070101-12","doi-asserted-by":"publisher","DOI":"10.1109\/eScience.2008.78"},{"key":"IJSIR.2018070101-13","doi-asserted-by":"publisher","DOI":"10.1109\/SIS.2005.1501606"},{"key":"IJSIR.2018070101-14","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-77465-5_9"},{"key":"IJSIR.2018070101-15","doi-asserted-by":"publisher","DOI":"10.1504\/IJCISTUDIES.2009.025340"},{"key":"IJSIR.2018070101-16","doi-asserted-by":"publisher","DOI":"10.1007\/s11721-008-0021-5"},{"key":"IJSIR.2018070101-17","unstructured":"Li, X., Engelbrecht, A., & Epitropakis, M. G. (2013). Benchmark Functions for CEC\u20192013 Special Session and Competition on Niching Methods for Multimodal Function Optimization. Retrieved from http:\/\/www.epitropakis.co.uk\/sites\/default\/files\/pubs\/cec2013-niching-benchmark-tech-report.pdf"},{"key":"IJSIR.2018070101-18","unstructured":"Liang, J. J., Qu, B. Y., Suganthan, P. N., & Chen, Q. (2014). Problem Definitions and Evaluation Criteria for the CEC 2015 Competition on Learning-based Real-Parameter Single Objective Optimization CEC 2015 Competition on Learning-based Real-Parameter Single Objective Optimization. Retrieved from https:\/\/pdfs.semanticscholar.org\/9693\/b1296203d421d9054d408e226f2e4d5ac15d.pdf"},{"key":"IJSIR.2018070101-19","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2007.4424448"},{"key":"IJSIR.2018070101-20","unstructured":"Meng, X., Bradley, J., Yavuz, B., Sparks, E., Venkataraman, S., & Liu, D. \u2026 Talwalkar, A. (2016). MLlib: Machine Learning in Apache Spark. Journal of Machine Learning Research, 17, 1\u20137. Retrieved from http:\/\/www.jmlr.org\/papers\/volume17\/15-237\/15-237.pdf"},{"key":"IJSIR.2018070101-21","unstructured":"Miryala, G. (2017). Comparative analysis of movie recommendation system using collaborative filtering in spark engine. Journal of Global Research in Computer Science, 8(10). Retrieved from http:\/\/www.jgrcs.info\/index.php\/jgrcs\/article\/view\/1015"},{"key":"IJSIR.2018070101-22","unstructured":"MLlib - Apache Spark. (2017). Retrieved November 16, 2017, from https:\/\/spark.apache.org\/mllib\/"},{"key":"IJSIR.2018070101-23","doi-asserted-by":"publisher","DOI":"10.1145\/3067695.3084219"},{"key":"IJSIR.2018070101-24","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2015.07.286"},{"key":"IJSIR.2018070101-25","unstructured":"Snir, M. (1998). MPI--the complete reference. MIT Press. Retrieved from https:\/\/mitpress.mit.edu\/books\/mpi-complete-reference-0"},{"key":"IJSIR.2018070101-26","unstructured":"Spark Streaming - Apache Spark. (2017). Retrieved November 16, 2017 from https:\/\/spark.apache.org\/streaming\/"},{"key":"IJSIR.2018070101-27","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-01020-0_2"},{"key":"IJSIR.2018070101-28","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-30976-2_22"},{"key":"IJSIR.2018070101-29","doi-asserted-by":"publisher","DOI":"10.2514\/1.17873"},{"key":"IJSIR.2018070101-30","doi-asserted-by":"publisher","DOI":"10.1109\/AMS.2008.27"},{"key":"IJSIR.2018070101-31","doi-asserted-by":"publisher","DOI":"10.1109\/ICNC.2012.6234645"},{"key":"IJSIR.2018070101-32","doi-asserted-by":"publisher","DOI":"10.3390\/app7040353"},{"key":"IJSIR.2018070101-33","unstructured":"Zaharia, M., Karau, H., & Konwinski, A. P. W. (2015). Learning Spark. O\u2019Reilly Media. Retrieved from http:\/\/shop.oreilly.com\/product\/0636920028512.do"},{"key":"IJSIR.2018070101-34","doi-asserted-by":"publisher","DOI":"10.1145\/1830483.1830689"}],"container-title":["International Journal of Swarm Intelligence Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.igi-global.com\/viewtitle.aspx?TitleId=210068","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,5,6]],"date-time":"2022-05-06T06:13:37Z","timestamp":1651817617000},"score":1,"resource":{"primary":{"URL":"http:\/\/services.igi-global.com\/resolvedoi\/resolve.aspx?doi=10.4018\/IJSIR.2018070101"}},"subtitle":["Glowworm Swarm Optimization Applied to Multimodal Functions"],"short-title":[],"issued":{"date-parts":[[2018,7]]},"references-count":35,"journal-issue":{"issue":"3"},"URL":"https:\/\/doi.org\/10.4018\/ijsir.2018070101","relation":{},"ISSN":["1947-9263","1947-9271"],"issn-type":[{"value":"1947-9263","type":"print"},{"value":"1947-9271","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7]]}}}