Abstract
Edge cloud technologies working in conjunction with AI-powered solutions can help surmount the challenges associated with the distributed execution of immersive services and contribute to delivering a positive end-user experience. Intelligent resource management, orchestration, and predictive systems can enhance service deployment, adapt to changing demands, and ensure seamless service operation. This paper introduces an innovative architectural paradigm that enables multi-domain edge orchestration for highly distributed immersive services by leveraging various AI solutions and technological tools to support multi-domain edge deployments. The proposed architecture is designed to function based on multi-level specification blueprints, separating high-level user-intent infrastructure definition from AI-driven orchestration and the final execution plan. This architectural design enables the incorporation of AI solutions to be conducted in a modular manner. Furthermore, the Application Management Framework provides a visual language and tool as an alternative to formal methods for creating intent blueprints. The proposed architecture is evaluated within the frame of an immersive virtual touring use-case scenario.

























Similar content being viewed by others
Data availability
Enquiries about data availability should be directed to the authors.
References
Makris, A., Boudi, A., Coppola, M., Cordeiro, L., Corsini, M., Dazzi, P., Andilla, F.D., González Rozas, Y., Kamarianakis, M., Pateraki, M., Pham, T.L., Protopsaltis, A., Raman, A., Romussi, A., Rosa, L., Spatafora, E., Taleb, T., Theodoropoulos, T., Tserpes, K., Zschau, E., Herzog, U.: Cloud for holography and augmented reality. In: 2021 IEEE 10th International Conference on Cloud Networking (CloudNet), pp. 118–126 (2021). https://doi.org/10.1109/CloudNet53349.2021.9657125
Taleb, T., Nadir, Z., Flinck, H., Song, J.: Extremely interactive and low-latency services in 5g and beyond mobile systems. IEEE Commun. Stand. Magn. 5(2), 114–119 (2021). https://doi.org/10.1109/MCOMSTD.001.2000053
Nadir, Z., Taleb, T., Flinck, H., Bouachir, O., Bagaa, M.: Immersive services over 5g and beyond mobile systems. IEEE Netw. 35(6), 299–306 (2021). https://doi.org/10.1109/MNET.121.2100172
Yu, H., Taleb, T., Samdanis, K., Song, J.: Towards supporting holographic services over deterministic 6g integrated terrestrial & non-terrestrial networks. IEEE Netw. (2023). https://doi.org/10.1109/MNET.133.2200509
Boos, K., Chu, D., Cuervo, E.: Demo: Flashback: Immersive virtual reality on mobile devices via rendering memorization. In: Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services Companion. MobiSys ’16 Companion, p. 94. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2938559.2938583
El Marai, O., Taleb, T., Song, J.: Ar-based remote command and control service: self-driving vehicles use case. IEEE Netw. 37(3), 170–177 (2023). https://doi.org/10.1109/MNET.119.2200058
Taleb, T., Sehad, N., Nadir, Z., Song, J.: Vr-based immersive service management in b5g mobile systems: a UAV command and control use case. IEEE Internet Things J. 10(6), 5349–5363 (2023). https://doi.org/10.1109/JIOT.2022.3222282
Theodoropoulos, T., Makris, A., Boudi, A., Taleb, T., Herzog, U., Rosa, L., Cordeiro, L., Tserpes, K., Spatafora, E., Romussi, A., et al.: Cloud-based XR services: a survey on relevant challenges and enabling technologies. J. Netw. Netw. Appl. 2(1), 1–22 (2022) https://doi.org/10.33969/J-NaNA.2022.020101
Taleb, T., Boudi, A., Rosa, L., Cordeiro, L., Theodoropoulos, T., Tserpes, K., Dazzi, P., Protopsaltis, A.I., Li, R.: Toward supporting XR services: architecture and enablers. IEEE Internet Things J. 10(4), 3567–3586 (2023). https://doi.org/10.1109/JIOT.2022.3222103
Theodoropoulos, T., Makris, A., Psomakelis, E., Carlini, E., Mordacchini, M., Dazzi, P., Tserpes, K.: Gnosis: proactive image placement using graph neural networks & deep reinforcement learning. In: 2023 IEEE 16th International Conference on Cloud Computing (CLOUD), pp. 120–128 (2023). https://doi.org/10.1109/CLOUD60044.2023.00022
Benmerar, T.Z., Theodoropoulos, T., Fevereiro, D., Rosa, L., Rodrigues, J., Taleb, T., Barone, P., Tserpes, K., Cordeiro, L.: Intelligent multi-domain edge orchestration for highly distributed immersive services: an immersive virtual touring use case. In: 2023 IEEE International Conference on Edge Computing and Communications (EDGE), pp. 381–392 (2023). https://doi.org/10.1109/EDGE60047.2023.00061
Faticanti, F., Savi, M., De Pellegrini, F., Siracusa, D.: Locality-aware deployment of application microservices for multi-domain fog computing. Comput. Commun. 203, 180–191 (2023). https://doi.org/10.1016/j.comcom.2023.02.012
3GPP. TS 23.558: Architecture for enabling Edge Applications. Technical Report (2023)
Alonso, J., Orue-Echevarria, L., Casola, V., Torre, A.I., Huarte, M., Osaba, E., Lobo, J.L.: Understanding the challenges and novel architectural models of multi-cloud native applications—a systematic literature review. J. Cloud Comput. 12(1), 1–34 (2023). https://doi.org/10.1186/s13677-022-00367-6
Raj, P., Raman, A.: Automated multi-cloud operations and container orchestration, pp. 185–218. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78637-7_9
Tomarchio, O., Calcaterra, D., Di Modica, G.: Cloud resource orchestration in the multi-cloud landscape: a systematic review of existing frameworks. J. Cloud Comput. 9, 49 (2020). https://doi.org/10.1186/s13677-020-00194-7
Bellendorf, J., Mann, Z.Á.: Specification of cloud topologies and orchestration using Tosca: a survey. Computing 102(8), 1793–1815 (2020). https://doi.org/10.1007/s00607-019-00750-3
Kim, D., Muhammad, H., Kim, E., Helal, S., Lee, C.: Tosca-based and federation-aware cloud orchestration for Kubernetes container platform. Appl. Sci. (2019). https://doi.org/10.3390/app9010191
Osmani, L., Kauppinen, T., Komu, M., Tarkoma, S.: Multi-cloud connectivity for Kubernetes in 5g networks. IEEE Commun. Magn. 59(10), 42–47 (2021). https://doi.org/10.1109/MCOM.110.2100124
Tamiru, M.A., Pierre, G., Tordsson, J., Elmroth, E.: mck8s: an orchestration platform for geo-distributed multi-cluster environments. In: 2021 International Conference on Computer Communications and Networks (ICCCN), pp. 1–10 (2021). https://doi.org/10.1109/ICCCN52240.2021.9522318
ETSI GS ZSM 011: Zero-touch network and Service Management (ZSM). Intent-driven autonomous networks; Generic aspects (2023)
Liyanage, M., Pham, Q.-V., Dev, K., Bhattacharya, S., Maddikunta, P.K.R., Gadekallu, T.R., Yenduri, G.: A survey on zero touch network and service management (ZSM) for 5g and beyond networks. J. Netw. Comput. Appl. 203, 103362 (2022). https://doi.org/10.1016/j.jnca.2022.103362
Coronado, E., Behravesh, R., Subramanya, T., Fernàndez-Fernàndez, A., Siddiqui, M.S., Costa-Pérez, X., Riggio, R.: Zero touch management: a survey of network automation solutions for 5g and 6g networks. IEEE Commun. Surv. Tutor. 24(4), 2535–2578 (2022). https://doi.org/10.1109/COMST.2022.3212586
Huang, S.-Y., Chen, C.-Y., Chen, J.-Y., Chao, H.-C.: A survey on resource management for cloud native mobile computing: opportunities and challenges. Symmetry (2023). https://doi.org/10.3390/sym15020538
Nejabati, R., Moazzeni, S., Jaisudthi, P., Simenidou, D.: Zero-touch network orchestration at the edge. In: 2021 International Conference on Computer Communications and Networks (ICCCN), pp. 1–5 (2021). https://doi.org/10.1109/ICCCN52240.2021.9522194
Gallego-Madrid, J., Sanchez-Iborra, R., Ruiz, P.M., Skarmeta, A.F.: Machine learning-based zero-touch network and service management: a survey. Digit. Commun. Netw. 8(2), 105–123 (2022). https://doi.org/10.1016/j.dcan.2021.09.001
Benzaid, C., Taleb, T.: Ai-driven zero touch network and service management in 5g and beyond: challenges and research directions. IEEE Netw. 34(2), 186–194 (2020). https://doi.org/10.1109/MNET.001.1900252
ETSI GS ZSM 012: Zero-touch network and Service Management (ZSM); Enablers for Artificial Intelligence-based Network and Service Automation (2022)
ETSI ZSM 008: Zero-touch network and Service Management (ZSM); Cross-domain E2E service lifecycle management (2022)
Korontanis, I., Tserpes, K., Pateraki, M., Blasi, L., Violos, J., Diego, F., Marin, E., Kourtellis, N., Coppola, M., Carlini, E., et al.: Inter-operability and orchestration in heterogeneous cloud/edge resources: the accordion vision. In: Proceedings of the 1st Workshop on Flexible Resource and Application Management on the Edge, pp. 9–14 (2020). https://doi.org/10.1145/3452369.3463816
3GPP. TR 28.312: Management and orchestration; Intent driven management services for mobile networks (2023)
3GPP. TR 28.912: Study on enhanced intent driven management services for mobile networks (2023)
3GPP. TR 28.812: Telecommunication management; Study on scenarios for Intent driven management services for mobile networks (2020)
Gutierrez-Estevez, D.M., Gramaglia, M., Domenico, A.D., Dandachi, G., Khatibi, S., Tsolkas, D., Balan, I., Garcia-Saavedra, A., Elzur, U., Wang, Y.: Artificial intelligence for elastic management and orchestration of 5g networks. IEEE Wirel. Commun. 26(5), 134–141 (2019). https://doi.org/10.1109/MWC.2019.1800498
Linux Foundation: ONAP—Open Network Automation Platform (2023). https://www.onap.org/. Accessed 02 May 2023
Linux Foundation: Akraino (2023). https://www.lfedge.org/projects/akraino/. Accessed 02 May 2023
Cluster API: Kubernetes Cluster API (2023). https://cluster-api.sigs.k8s.io/. Accessed 02 May 2023
ETSI: OSM—Open Source MANO (2023). https://osm.etsi.org/. Accessed 02 May 2023
Cloudify: Bridging the gap between applications and cloud environments (2023). https://cloudify.co/. Accessed 02 May 2023
Redhat: Redhat—Openshift (2023). https://www.redhat.com/en/technologies/cloud-computing/openshift. Accessed 02 May 2023
Tamburri, D.A., Heuvel, W.-J., Lauwers, C., Lipton, P., Palma, D., Rutkowski, M.: Tosca-based intent modelling: goal-modelling for infrastructure-as-code. SICS Softw. Intensive Cyber-Phys. Syst. 34(2), 163–172 (2019). https://doi.org/10.1007/s00450-019-00404-x
Theodoropoulos, T., Makris, A., Kontopoulos, I., Maroudis, A.-C., Tserpes, K.: Multi-service demand forecasting using graph neural networks. In: 2023 IEEE International Conference on Service-Oriented System Engineering (SOSE), pp. 218–226 (2023). https://doi.org/10.1109/SOSE58276.2023.00033
Yilmaz, O.: Extending the Kubernetes API, pp. 99–141. Apress, Berkeley (2021). https://doi.org/10.1007/978-1-4842-7095-0_4
Lim, B., Zohren, S.: Time-series forecasting with deep learning: a survey. Philos. Trans. R. Soc. A 379(2194), 1–14 (2021). https://doi.org/10.1098/rsta.2020.0209
Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: Deep reinforcement learning: a brief survey. IEEE Signal Process. Magn. 34(6), 26–38 (2017). https://doi.org/10.1109/MSP.2017.2743240
Theodoropoulos, T., Maroudis, A.-C., Violos, J., Tserpes, K.: An encoder-decoder deep learning approach for multistep service traffic prediction. In: 2021 IEEE Seventh International Conference on Big Data Computing Service and Applications (BigDataService), pp. 33–40 (2021). https://doi.org/10.1109/BigDataService52369.2021.00010
Theodoropoulos, T., Makris, A., Kontopoulos, I., Violos, J., Tarkowski, P., Ledwoń, Z., Dazzi, P., Tserpes, K.: Graph neural networks for representing multivariate resource usage: a multiplayer mobile gaming case-study. Int. J. Inf. Manag. Data Insights 3(1), 100158 (2023). https://doi.org/10.1016/j.jjimei.2023.100158
Fang, C., Zhang, T., Huang, J., Xu, H., Hu, Z., Yang, Y., Wang, Z., Zhou, Z., Luo, X.: A DRL-driven intelligent optimization strategy for resource allocation in cloud-edge-end cooperation environments. Symmetry (2022). https://doi.org/10.3390/sym14102120
Zhang, Y., Li, Y., Wang, R., Lu, J., Ma, X., Qiu, M.: PSAC: proactive sequence-aware content caching via deep learning at the network edge. IEEE Trans. Netw. Sci. Eng. 7(4), 2145–2154 (2020). https://doi.org/10.1109/TNSE.2020.2990963
Behravesh, R., Rao, A., Perez-Ramirez, D.F., Harutyunyan, D., Riggio, R., Boman, M.: Machine learning at the mobile edge: the case of dynamic adaptive streaming over http (DASH). IEEE Trans. Netw. Serv. Manage. 19(4), 4779–4793 (2022). https://doi.org/10.1109/TNSM.2022.3193856
Narayanan, A., Verma, S., Ramadan, E., Babaie, P., Zhang, Z.-L.: Deepcache: A deep learning based framework for content caching. In: Proceedings of the 2018 Workshop on Network Meets AI & ML. NetAI’18, pp. 48–53. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3229543.3229555
Theodoropoulos, T., Kafetzis, D., Violos, J., Makris, A., Tserpes, K.: Multi-agent deep reinforcement learning for weighted multi-path routing. In: Proceedings of the 3rd Workshop on Flexible Resource and Application Management on the Edge. FRAME ’23, pp. 7–11. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3589010.3594888
Theodoropoulos, T., Makris, A., Violos, J., Tserpes, K.: An automated pipeline for advanced fault tolerance in edge computing infrastructures. In: Proceedings of the 2nd Workshop on Flexible Resource and Application Management on the Edge. FRAME ’22, pp. 19–24. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3526059.3533623
Ma, W.: Analysis of anomaly detection method for internet of things based on deep learning. Trans. Emerg. Telecommun. Technol. 31(12), 3893 (2020). https://doi.org/10.1002/ett.3893
Theodoropoulos, T., Violos, J., Tsanakas, S., Leivadeas, A., Tserpes, K., Varvarigou, T.: Intelligent proactive fault tolerance at the edge through resource usage prediction. ITU J. Future Evol. Technol. 3(3), 761–778 (2022). https://doi.org/10.52953/ehjp3291
Chen, W., Chen, Y., Wu, J., Tang, Z.: A multi-user service migration scheme based on deep reinforcement learning and SDN in mobile edge computing. Phys. Commun. 47, 101397 (2021). https://doi.org/10.1016/j.phycom.2021.101397
Al-Asaly, M.S., Bencherif, M.A., Alsanad, A., Hassan, M.M.: A deep learning-based resource usage prediction model for resource provisioning in an autonomic cloud computing environment. Neural Comput. Appl. (2021). https://doi.org/10.1007/s00521-021-06665-5
Xiao, Z., Hu, S.: Dscaler: A horizontal autoscaler of microservice based on deep reinforcement learning. In: 2022 23rd Asia-Pacific Network Operations and Management Symposium (APNOMS), pp. 1–6 (2022). https://doi.org/10.23919/APNOMS56106.2022.9919994
Violos, J., Tsanakas, S., Theodoropoulos, T., Leivadeas, A., Tserpes, K., Varvarigou, T.: Intelligent horizontal autoscaling in edge computing using a double tower neural network. Comput. Netw. 217, 109339 (2022). https://doi.org/10.1016/j.comnet.2022.109339
Liu, Q., Xia, T., Cheng, L., Eijk, M., Ozcelebi, T., Mao, Y.: Deep reinforcement learning for load-balancing aware network control in IoT edge systems. IEEE Trans. Parallel Distrib. Syst. 33(6), 1491–1502 (2022). https://doi.org/10.1109/TPDS.2021.3116863
Theodoropoulos, T., Makris, A., Korontanis, I., Tserpes, K.: Greenkube: Towards greener container orchestration using artificial intelligence. In: 2023 IEEE International Conference on Service-Oriented System Engineering (SOSE), pp. 135–139 (2023). https://doi.org/10.1109/SOSE58276.2023.00023
Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., Gao, Y.: A survey on federated learning. Knowl. Based Syst. 216, 106775 (2021). https://doi.org/10.1016/j.knosys.2021.106775
Li, L., Fan, Y., Tse, M., Lin, K.-Y.: A review of applications in federated learning. Comput. Ind. Eng. 149, 106854 (2020). https://doi.org/10.1016/j.cie.2020.106854
Chen, J., Ran, X.: Deep learning with edge computing: a review. Proc. IEEE 107(8), 1655–1674 (2019). https://doi.org/10.1109/JPROC.2019.2921977
Wang, Y., Guo, L., Zhao, Y., Yang, J., Adebisi, B., Gacanin, H., Gui, G.: Distributed learning for automatic modulation classification in edge devices. IEEE Wirel. Commun. Lett. 9(12), 2177–2181 (2020). https://doi.org/10.1109/LWC.2020.3016822
Cloud Native Computing Foundation: Prometheus (2023). https://prometheus.io. Accessed 02 May 2023
Korontanis, I., Makris, A., Theodoropoulos, T., Tserpes, K.: Real-time monitoring and analysis of edge and cloud resources. In: Proceedings of the 3rd Workshop on Flexible Resource and Application Management on the Edge. FRAME ’23, pp. 13–18. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3589010.3594892
Iorio, M., Risso, F., Palesandro, A., Camiciotti, L., Manzalini, A.: Computing without borders: the way towards liquid computing. IEEE Trans. Cloud Comput. 11(3), 2820–2838 (2023). https://doi.org/10.1109/TCC.2022.3229163
Cyango: Cyango—virtual reality, AR & Digital Transformation Studio (2023). https://www.cyango.com/. Accessed 8 Dec 2023
OASIS: Tosca simple profile version 1.3 (2020). https://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.3/os/TOSCA-Simple-Profile-YAML-v1.3-os.pdf
Peermetrics: Peermetrics (2023). https://github.com/peermetrics/webrtc-stats. Accessed 16 Oct 2023
Acknowledgements
This paper reflects only the authors’ view and the Commission is not responsible for any use that may be made of the information it contains.
Funding
This research work has been supported by the CHARITY project that received funding from the EU’s Horizon 2020 program under Grant agreement No 101016509.
Author information
Authors and Affiliations
Contributions
T.B and T.T. contributed to the theoretical work of the orchestration system global architecture and components work in the text. L.R., D.F. and L.C. contributed to the underlying technologies of the orchestration system (ClusterAPI and Liqo) sections. They also played a central role in the experimental setup and report in the manustript. J.R. contributed with the immersive use case sections and its metrics aggregation. P.B and G.G. contributed with AMF parts in the manuscript as well as the operations flows. T.T. and K.T. contributed with the AI-related parts in the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Benmerar, T.Z., Theodoropoulos, T., Fevereiro, D. et al. Towards establishing intelligent multi-domain edge orchestration for highly distributed immersive services: a virtual touring use case. Cluster Comput 27, 4223–4253 (2024). https://doi.org/10.1007/s10586-024-04413-7
Received:
Revised:
Accepted:
Published:
Version of record:
Issue date:
DOI: https://doi.org/10.1007/s10586-024-04413-7
