Skip to main content
Springer Nature Link
Log in

Parallel stacked autoencoder with particle swarm optimization for community detection in complex networks

  • Published:
Save article
View saved research
Applied Intelligence Aims and scope Submit manuscript

Abstract

Community detection is one of the long standing and challenging tasks in the field of Complex Networks (CNs). Recently, deep learning is one of the promising community detection methods, which can learn effectively low-dimensional representation of CNs. However, the existing methods have major drawbacks in terms of local minima and slow convergence, since they use Gradient Descent Backpropagation algorithm (GDBP). This reduces the performance of community detection in terms of effectiveness and efficiency. To overcome these drawbacks, this paper introduces a new parallel deep learning model based on Metaheuristic (MH) algorithm instead of the GDBP algorithm. To be specific, a new parallel stacked autoencoder (SAE) based on particle swarm optimization (PSO) is developed for feature learning and community detection in CNs. The PSO algorithm uses a multi-objective fitness function that includes the standard loss function (i.e., MSE) of the autoencoder and the modularity function to guide SAE optimization and improve community detection performance. In addition, an efficient distributed parallel implementation is proposed to improve the efficiency and scalability of the SAE-based PSO method. The parameter settings of PSO such as features-dimension and number of particles, are tuned and studied to observe their implications on community detection performance. We conducted an experiment comprising datasets of 10 real-world networks to evaluate the proposed method in different parameter settings. The results demonstrated that the SAE-based PSO method is promising and provides a competitive performance against state-of-art methods in community detection. Furthermore, the results showed that the parallel implementation of the proposed method could improve efficiency with three or greater orders of speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from €37.37 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price includes VAT (Netherlands)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
The alternative text for this image may have been generated using AI.
Fig. 2
The alternative text for this image may have been generated using AI.
Fig. 3
The alternative text for this image may have been generated using AI.
Fig. 4
The alternative text for this image may have been generated using AI.
Fig. 5
The alternative text for this image may have been generated using AI.
Fig. 6
The alternative text for this image may have been generated using AI.
Fig. 7
The alternative text for this image may have been generated using AI.
Fig. 8
The alternative text for this image may have been generated using AI.
Fig. 9
The alternative text for this image may have been generated using AI.
Fig. 10
The alternative text for this image may have been generated using AI.

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Notes

  1. https//github.com/zepx

  2. yangliang.github.io/code/DC.zip

  3. https//github.com/smartyfh/DANMF

  4. https//github.com/dilberdillu/community-detection-DL

References

  1. Xu R, Che Y, Wang X, Hu J, Xie Y (2020) Stacked Autoencoder-Based Community Detection Method via an Ensemble Clustering Framework. Inf Sci. https://doi.org/10.1016/j.ins.2020.03.090

  2. Javed MA, Younis MS, Latif S, Qadir J, Baig A (2018) Community detection in networks: A multidisciplinary review. J Netw Comput Appl 108:87–111. https://doi.org/10.1016/j.jnca.2018.02.011

    Article  Google Scholar 

  3. Moradi P, Rezaimehr F, Ahmadian S, Jalili M (2016) A trust-aware recommender algorithm based on users overlapping community structure, in 2016 Sixteenth International Conference on Advances in ICT for Emerging Regions (ICTer), IEEE, pp. 162–167

  4. Ahmadian S, Meghdadi M, Afsharchi M (2018) Incorporating reliable virtual ratings into social recommendation systems. Appl Intell 48(11):4448–4469

    Article  Google Scholar 

  5. Bonchi F, Castillo C, Gionis A, Jaimes A (2011) Social network analysis and mining for business applications. ACM Trans Intell Syst Technol (TIST) 2(3):22. https://doi.org/10.1145/1961189.1961194

    Article  Google Scholar 

  6. Khatoon M, Banu WA (2015) A survey on community detection methods in social networks. Intl J Educ Manag Eng 5(1):8. https://doi.org/10.5815/ijeme.2015.01.02

    Article  Google Scholar 

  7. Ng AY, Jordan MI, Weiss Y (2002) On spectral clustering: Analysis and an algorithm, in Advances in neural information processing systems, pp. 849–856

  8. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations, in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, pp. 701–710, doi: https://doi.org/10.1145/2623330.2623732

  9. Jalali SMJ et al (2021) Towards novel deep neuroevolution models: chaotic levy grasshopper optimization for short-term wind speed forecasting. Eng Comput:1–25

  10. Pinaya WHL, Vieira S, Garcia-Dias R, Mechelli A (2020) utoencoders, in Machine Learning: Elsevier, pp. 193–208

  11. Yang L, Cao X, He D, Wang C, Wang X, Zhang W (2016) Modularity Based Community Detection with Deep Learning, in IJCAI, vol. 16, pp. 2252–2258

  12. Tian F, Gao B, Cui Q, Chen E, Liu T-Y (2014) Learning deep representations for graph clustering, in twenty-eighth AAAI conference on artificial intelligence

  13. Ye F, Chen C, Zheng Z (2018) Deep autoencoder-like nonnegative matrix factorization for community detection, in Proceedings of the 27th ACM International Conference on Information and Knowledge Management, ACM, pp. 1393–1402, doi: https://doi.org/10.1145/3269206.3271697

  14. Fei R, Sha J, Xu Q, Hu B, Wang K, Li S (2020) A new deep sparse autoencoder for community detection in complex networks, EURASIP Journal on Wireless Communications and Networking, vol. pp. 1–25, 2020, doi: https://doi.org/10.1186/s13638-020-01706-4

  15. He T, Cai L, Meng T, Chen L, Deng Z, Cao Z (2018) Parallel community detection based on distance dynamics for large-scale network. IEEE Access 6:42775–42789. https://doi.org/10.1109/ACCESS.2018.2859788

    Article  Google Scholar 

  16. S. Moon, J.-G. Lee, M. Kang, M. Choy, and J.-w. Lee, "Parallel community detection on large graphs with MapReduce and GraphChi," Data Knowledge Eng, vol. 104, pp. 17–31, 2016, doi: https://doi.org/10.1016/j.datak.2015.05.001

  17. Al-Ayyoub M, Al-andoli M, Jararweh Y, Smadi M, Gupta B (2019) Improving fuzzy C-mean-based community detection in social networks using dynamic parallelism. Comput Electric Eng 74:533–546. https://doi.org/10.1016/j.compeleceng.2018.01.003

    Article  Google Scholar 

  18. Fong S, Deb S, Yang X-s (2018) How meta-heuristic algorithms contribute to deep learning in the hype of big data analytics, in Progress in Intelligent Computing Techniques: Theory, Practice, and Applications: Springer, pp. 3–25

  19. Ahmadian S, Khanteymoori AR (2015) Training back propagation neural networks using asexual reproduction optimization, in 2015 7th Conference on Information and Knowledge Technology (IKT), IEEE, pp. 1–6

  20. Yu J-B (2019) Evolutionary manifold regularized stacked denoising autoencoders for gearbox fault diagnosis. Knowledge-Based Syst 178:111–122. https://doi.org/10.1016/j.knosys.2019.04.022

    Article  Google Scholar 

  21. Tian Z, Fong S (2016) Survey of meta-heuristic algorithms for deep learning training, Optimization algorithms—methods and applications

  22. Gudise VG, Venayagamoorthy GK (2003) Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks, in Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS ' 03 (Cat. No. 03EX706), IEEE, pp. 110–117, doi: https://doi.org/10.1109/SIS.2003.1202255

  23. Zhang J-R, Zhang J, Lok T-M, Lyu MR (2007) A hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. Appl math comput 185(2):1026–1037. https://doi.org/10.1016/j.amc.2006.07.025

    Article  MATH  Google Scholar 

  24. Geng W (2018) Cognitive Deep Neural Networks prediction method for software fault tendency module based on Bound Particle Swarm Optimization. Cognit Syst Res 52:12–20. https://doi.org/10.1016/j.cogsys.2018.06.001

    Article  Google Scholar 

  25. Sun Y, Xue B, Zhang M, Yen GG (2018) A particle swarm optimization-based flexible convolutional autoencoder for image classification. IEEE Trans Neural Netw Learn Syst 30(8):2295–2309. https://doi.org/10.1109/TNNLS.2018.2881143

    Article  Google Scholar 

  26. Moritz P et al. (2018) Ray: A distributed framework for emerging {AI} applications, in 13th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 18), pp. 561–577

  27. Newman MEJ, Girvan M (2004) Finding and evaluating community structure in networks. Phys Rev E 69(2). https://doi.org/10.1103/physreve.69.026113

  28. Freeman LC (1977) A set of measures of centrality based on betweenness. Sociometry 40:35–41. https://doi.org/10.2307/3033543

    Article  Google Scholar 

  29. Raghavan UN, Albert R, Kumara S (2007) Near linear time algorithm to detect community structures in large-scale networks. Phys Rev E 76(3):036106. https://doi.org/10.1103/PhysRevE.76.036106

    Article  Google Scholar 

  30. Soman J, Narang A (2011) Fast community detection algorithm with gpus and multicore architectures, in 2011 IEEE International Parallel & Distributed Processing Symposium, IEEE, pp. 568–579, doi: https://doi.org/10.1109/IPDPS.2011.61

  31. M. Al-Andoli, W. P. Cheah, and S. C. Tan, Deep learning-based community detection in complex networks with network partitioning and reduction of trainable parameters," J Ambient Intell Human Comput, pp. 1–19, 2020, doi: https://doi.org/10.1007/s12652-020-02389-x

  32. Dhilber M, Bhavani SD (2020) Community Detection in Social Networks Using Deep Learning, in International Conference on Distributed Computing and Internet Technology, Springer, pp. 241–250

  33. Wu L, Zhang Q, Chen C-H, Guo K, Wang D (2020) Deep Learning Techniques for Community Detection in Social Networks, IEEE Access, doi: https://doi.org/10.1109/ACCESS.2020.2996001

  34. Said A, Abbasi RA, Maqbool O, Daud A, Aljohani NR (2018) CC-GA: a clustering coefficient based genetic algorithm for detecting communities in social networks. Appl Soft Comput 63:59–70

    Article  Google Scholar 

  35. Zarei B, Meybodi MR (2020) Detecting community structure in complex networks using genetic algorithm based on object migrating automata. Comput Intell 36(2):824–860

    Article  Google Scholar 

  36. Cai Q, Gong M, Ma L, Ruan S, Yuan F, Jiao L (2015) Greedy discrete particle swarm optimization for large-scale social network clustering. Inf Sci 316:503–516

    Article  Google Scholar 

  37. Liu X, Du Y, Jiang M, Zeng X (2020) Multiobjective Particle Swarm Optimization Based on Network Embedding for Complex Network Community Detection. IEEE Trans Comput Soc Syst 7(2):437–449. https://doi.org/10.1109/TCSS.2020.2964027

    Article  Google Scholar 

  38. Del Ser J, Lobo JL, Villar-Rodriguez E, Bilbao MN, Perfecto C (2016) Community detection in graphs based on surprise maximization using firefly heuristics, in 2016 IEEE congress on evolutionary computation (CEC), IEEE, pp. 2233–2239

  39. Y. Guo, X. Li, Y. Tang, and J. Li (2017) Heuristic artificial bee colony algorithm for uncovering community in complex networks, Math Prob Eng, vol 2017

  40. Sani NS, Manthouri M, Farivar F (2020) A multi-objective ant colony optimization algorithm for community detection in complex networks. J Ambient Intell Humaniz Comput 11(1):5–21. https://doi.org/10.1007/s12652-018-1159-7

    Article  Google Scholar 

  41. Zhou X, Liu Y, Li B, Li H (2017) A multiobjective discrete cuckoo search algorithm for community detection in dynamic networks. Soft Comput 21(22):6641–6652

    Article  Google Scholar 

  42. Barham R, Sharieh A, Sleit A (2019) Multi-moth flame optimization for solving the link prediction problem in complex networks. Evol Intel 12(4):563–591

    Article  Google Scholar 

  43. Srilatha P, Manjula R (2017) Structural similarity based link prediction in social networks using firefly algorithm, in 2017 International Conference On Smart Technologies For Smart Nation (SmartTechCon), IEEE, pp. 560–564

  44. Barham R, Aljarah I (2017) Link prediction based on whale optimization algorithm, in 2017 International Conference on New Trends in Computing Sciences (ICTCS), IEEE, pp. 55–60

  45. Sahoo SR, Gupta BB (2020) Classification of spammer and nonspammer content in online social network using genetic algorithm-based feature selection. Enterp Inf Syst 14(5):710–736

    Article  Google Scholar 

  46. Carneiro MG, Cheng R, Zhao L, Jin Y (2019) Particle swarm optimization for network-based data classification. Neural Netw 110:243–255

    Article  Google Scholar 

  47. Akyol S, Alatas B (2020) Sentiment classification within online social media using whale optimization algorithm and social impact theory based optimization. Phys A: Stat Mech Appl 540:123094

    Article  Google Scholar 

  48. Abduljabbar DA, Hashim SZM, Sallehuddin R (2020) Nature-inspired optimization algorithms for community detection in complex networks: a review and future trends. Telecommun Syst 74(2):225–252

    Article  Google Scholar 

  49. Bhattacharya M (2013) Expensive optimisation: A metaheuristics perspective, arXiv preprint arXiv:1303.2215

  50. Hegde S, Mundada MR (2018) A Hybrid Approach of Deep Learning with Cognitive Particle Swarm Optimization for the Big Data Analytics, in 2018 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT), IEEE, pp. 1–5, doi: https://doi.org/10.1109/ICCCNT.2018.8494200

  51. Cao J, Cui H, Shi H, Jiao L (2016) Big data: A parallel particle swarm optimization-back-propagation neural network algorithm based on MapReduce. PloS one 11(6):e0157551. https://doi.org/10.1371/journal.pone.0157551

    Article  Google Scholar 

  52. Saranya C, Nagarajan N (2020) Efficient agricultural yield prediction using metaheuristic optimized artificial neural network using Hadoop framework, Soft Comput, pp. 1–11, doi: https://doi.org/10.1007/s00500-020-04707-z

  53. Xie Y, Gong M, Wang S, Yu B (2018) Community discovery in networks with deep sparse filtering. Pattern Recogn 81:50–59. https://doi.org/10.1016/j.patcog.2018.03.026

    Article  Google Scholar 

  54. Jaccard P (1912) The distribution of the flora in the alpine zone. 1. New Phytol 11(2):37–50

    Article  Google Scholar 

  55. SpolaôR N, Cherman EA, Monard MC, Lee HD (2013) A comparison of multi-label feature selection methods using the problem transformation approach. Electron Notes Theor Comput Sci 292:135–151

    Article  Google Scholar 

  56. Kennedy J (2010) Particle swarm optimization, Encyclopedia of machine learning, pp. 760–766

  57. Zachary WW (1977) An information flow model for conflict and fission in small groups. J Anthropol Res 33(4):452–473. https://doi.org/10.1086/jar.33.4.3629752

    Article  Google Scholar 

  58. Lusseau D, Schneider K, Boisseau OJ, Haase P, Slooten E, Dawson SM (2003) The bottlenose dolphin community of Doubtful Sound features a large proportion of long-lasting associations. Behav Ecol Sociobiol 54(4):396–405. https://doi.org/10.1007/s00265-003-0651-y

    Article  Google Scholar 

  59. Xie J, Kelley S, Szymanski BK (2013) Overlapping community detection in networks: The state-of-the-art and comparative study. ACM Comput Surv (csur) 45(4):1–35. https://doi.org/10.1145/2501654.2501657

    Article  MATH  Google Scholar 

  60. Girvan M, Newman ME (2002) Community structure in social and biological networks. Proc Natl Acad Sci 99(12):7821–7826. https://doi.org/10.1073/pnas.122653799

    Article  MathSciNet  MATH  Google Scholar 

  61. Newman ME (2006) Modularity and community structure in networks. Proc Natl Acad Sci 103(23):8577–8582. https://doi.org/10.1073/pnas.0601602103

    Article  Google Scholar 

  62. Adamic LA, Glance N (2005) The political blogosphere and the 2004 US election: divided they blog, in Proceedings of the 3rd international workshop on Link discovery, pp. 36–43, doi: https://doi.org/10.1145/1134271.1134277. [Online]. Available: https://dl.acm.org/doi/abs/10.1145/1134271.1134277

  63. Yang T, Jin R, Chi Y, Zhu S (2009) Combining link and content for community detection: a discriminative approach, in Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 927–936, doi: https://doi.org/10.1145/1557019.1557120

  64. L. Danon, A. Diaz-Guilera, J. Duch, and A. Arenas (2005) Comparing community structure identification, J Stat Mech: Theory and Experiment, (09):09x008

  65. Hu L, Gorton I (1997) Performance evaluation for parallel systems: a survey. Citeseer

  66. Salimans T, Ho J, Chen X, Sidor S, Sutskever I (2017) Evolution strategies as a scalable alternative to reinforcement learning, arXiv preprint arXiv:1703.03864

  67. Pierezan J, Coelho LDS (2018) Coyote optimization algorithm: a new metaheuristic for global optimization problems, in 2018 IEEE congress on evolutionary computation (CEC), IEEE, pp. 1–8

  68. Katiyar S (2010) A comparative study of genetic algorithm and the particle swarm optimization. Intl J Technol 2(2):21–24

    Google Scholar 

  69. Demšar, Statistical comparisons of classifiers over multiple data sets, J Mach Learn Res 7 (2006) 1–30

  70. Vavasis SA (2010) On the complexity of nonnegative matrix factorization. SIAM J Optim 20(3):1364–1377

    Article  MathSciNet  Google Scholar 

  71. Ojha VK, Abraham A, Snášel V (2017) Metaheuristic design of feedforward neural networks: a review of two decades of research. Eng Appl Artif Intell 60:97–116

    Article  Google Scholar 

  72. Wu CW (2018) ProdSumNet: reducing model parameters in deep neural networks via product-of-sums matrix decompositions, arXiv preprint arXiv:1809.02209

Download references

Acknowledgements

This work is supported financially by the FRGS from the Ministry of Higher Education and Multimedia University, Malaysia (Project ID: FRGS/1/2018/ICT02/MMU/02/1).

Author information

Authors and Affiliations

  1. Faculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Bukit Beruang, Melaka, Malaysia

    Mohammed Al-Andoli & Shing Chiang Tan

  2. School of Computer Science, Faculty of Science and Engineering, University of Nottingham Ningbo China, Taikang East Road, Ningbo, 315100, China

    Wooi Ping Cheah

Authors
  1. Mohammed Al-Andoli
    View author publications

    Search author on:PubMed Google Scholar

  2. Shing Chiang Tan
    View author publications

    Search author on:PubMed Google Scholar

  3. Wooi Ping Cheah
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Shing Chiang Tan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al-Andoli, M., Tan, S.C. & Cheah, W.P. Parallel stacked autoencoder with particle swarm optimization for community detection in complex networks. Appl Intell 52, 3366–3386 (2022). https://doi.org/10.1007/s10489-021-02589-8

Download citation

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s10489-021-02589-8

Keywords