{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T06:56:30Z","timestamp":1762066590099,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,15]],"date-time":"2022-09-15T00:00:00Z","timestamp":1663200000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62271068","61827801","NS2022046"],"award-info":[{"award-number":["62271068","61827801","NS2022046"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Basic Scientific Research Project","award":["62271068","61827801","NS2022046"],"award-info":[{"award-number":["62271068","61827801","NS2022046"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Mobile edge computing (MEC) and device-to-device (D2D) communication can alleviate the resource constraints of mobile devices and reduce communication latency. In this paper, we construct a D2D-MEC framework and study the multi-user cooperative partial offloading and computing resource allocation. We maximize the number of devices under the maximum delay constraints of the application and the limited computing resources. In the considered system, each user can offload its tasks to an edge server and a nearby D2D device. We first formulate the optimization problem as an NP-hard problem and then decouple it into two subproblems. The convex optimization method is used to solve the first subproblem, and the second subproblem is defined as a Markov decision process (MDP). A deep reinforcement learning algorithm based on a deep Q network (DQN) is developed to maximize the amount of tasks that the system can compute. Extensive simulation results demonstrate the effectiveness and superiority of the proposed scheme.<\/jats:p>","DOI":"10.3390\/s22187004","type":"journal-article","created":{"date-parts":[[2022,9,16]],"date-time":"2022-09-16T01:35:10Z","timestamp":1663292110000},"page":"7004","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["D2D-Assisted Multi-User Cooperative Partial Offloading in MEC Based on Deep Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"given":"Xin","family":"Guan","sequence":"first","affiliation":[{"name":"School of Information and Communication Engineering, Beijing University of Posts and Telecommunications (BUPT), Beijing 100876, China"}]},{"given":"Tiejun","family":"Lv","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, Beijing University of Posts and Telecommunications (BUPT), Beijing 100876, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6111-9687","authenticated-orcid":false,"given":"Zhipeng","family":"Lin","sequence":"additional","affiliation":[{"name":"Key Laboratory of Dynamic Cognitive System of Electromagnetic Spectrum Space, College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics (NUAA), Nanjing 211106, China"}]},{"given":"Pingmu","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Beijing University of Posts and Telecommunications (BUPT), Beijing 100876, China"}]},{"given":"Jie","family":"Zeng","sequence":"additional","affiliation":[{"name":"School of Cyberspace Science and Technology, Beijing Institute of Technology, Beijing 100081, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Mangiante, S., Klas, G., Navon, A., GuanHua, Z., Ran, J., and Silva, M.D. (2017, January 8\u201311). Vr is on the edge: How to deliver 360 videos in mobile networks. Proceedings of the Workshop on Virtual Reality and Augmented Reality Network, New York, NY, USA.","DOI":"10.1145\/3097895.3097901"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Yang, Z., Liu, Y., Chen, Y., and Tyson, G. (2019, January 20\u201324). Deep reinforcement learning in cache-aided MEC networks. Proceedings of the ICC 2019\u20142019 IEEE International Conference on Communications (ICC), Shanghai, China.","DOI":"10.1109\/ICC.2019.8761349"},{"key":"ref_3","first-page":"3571","article-title":"Offloading in Mobile Edge Computing: Task Allocation and Computational Frequency Scaling","volume":"65","author":"Dinh","year":"2017","journal-title":"IEEE Trans. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"103366","DOI":"10.1016\/j.jnca.2022.103366","article-title":"Computation offloading in mobile edge computing networks: A survey","volume":"202","author":"Feng","year":"2022","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2410","DOI":"10.1109\/TNSM.2020.3020249","article-title":"Efficient and secure multi-user multi-task computation offloading for mobile-edge computing in mobile IoT networks","volume":"17","author":"Elgendy","year":"2020","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"137052","DOI":"10.1109\/ACCESS.2020.3011705","article-title":"Advanced deep learning-based computational offloading for multilevel vehicular edge-cloud computing networks","volume":"8","author":"Khayyat","year":"2020","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"8119","DOI":"10.1109\/JIOT.2020.3042433","article-title":"Secure and optimized load balancing for multitier IoT and edge-cloud computing systems","volume":"8","author":"Zhang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1657","DOI":"10.1109\/COMST.2017.2705720","article-title":"On multi-access edge computing: A survey of the emerging 5G network edge cloud architecture and orchestration","volume":"19","author":"Taleb","year":"2017","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"102974","DOI":"10.1016\/j.jnca.2021.102974","article-title":"An autonomous computation offloading strategy in Mobile Edge Computing: A deep learning-based hybrid approach","volume":"178","author":"Shakarami","year":"2021","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1628","DOI":"10.1109\/COMST.2017.2682318","article-title":"Mobile edge computing: A survey on architecture and computation offloading","volume":"19","author":"Mach","year":"2017","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1109\/TSC.2021.3116280","article-title":"An efficient online computation offloading approach for large-scale mobile edge computing via deep reinforcement learning","volume":"15","author":"Hu","year":"2021","journal-title":"IEEE Trans. Serv. Comput."},{"key":"ref_12","unstructured":"Kumari, P., Mishra, R., Gupta, H.P., Dutta, T., and Das, S.K. (2021). An energy efficient smart metering system using edge computing in LoRa network. IEEE Trans. Sustain. Comput., 1\u201313."},{"key":"ref_13","first-page":"393","article-title":"A survey of mobile cloud computing application models","volume":"16","author":"Othman","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3962","DOI":"10.1109\/ACCESS.2018.2791504","article-title":"Multi-objective decision-making for mobile cloud offloading: A survey","volume":"6","author":"Wu","year":"2018","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Chalaemwongwan, N., and Kurutach, W. (July, January 28). Mobile cloud computing: A survey and propose solution framework. Proceedings of the 2016 13th International Conference on Electrical Engineering\/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), Chiang Mai, Thailand.","DOI":"10.1109\/ECTICon.2016.7561437"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, J., Gao, H., Lv, T., and Lu, Y. (2018, January 15\u201318). Deep reinforcement learning based computation offloading and resource allocation for MEC. Proceedings of the 2018 IEEE Wireless communications and networking conference (WCNC), Barcelona, Spain.","DOI":"10.1109\/WCNC.2018.8377343"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2023","DOI":"10.1007\/s11276-021-02554-w","article-title":"Joint computation offloading and task caching for multi-user and multi-task MEC systems: Reinforcement learning-based algorithms","volume":"27","author":"Elgendy","year":"2021","journal-title":"Wirel. Netw."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1133","DOI":"10.1109\/TPDS.2020.3042224","article-title":"Multi-hop multi-task partial computation offloading in collaborative edge computing","volume":"32","author":"Sahni","year":"2020","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Chouhan, S. (2019, January 19\u201321). Energy optimal partial computation offloading framework for mobile devices in multi-access edge computing. Proceedings of the 2019 International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Croatia.","DOI":"10.23919\/SOFTCOM.2019.8903763"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4858","DOI":"10.1109\/TWC.2021.3062616","article-title":"D2D-assisted multi-user cooperative partial offloading, transmission scheduling and computation allocating for MEC","volume":"20","author":"Peng","year":"2021","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ti, N.T., and Le, L.B. (2017, January 21\u201325). Computation offloading leveraging computing resources from edge cloud and mobile peers. Proceedings of the 2017 IEEE International Conference on Communications (ICC), Paris, France.","DOI":"10.1109\/ICC.2017.7997138"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wang, C., Qin, J., Yang, X., and Wen, W. (2019, January 8\u201310). Energy-efficient offloading policy in D2D underlay communication integrated with MEC service. Proceedings of the 3rd International Conference on High Performance Compilation, Computing and Communications, Xi\u2019an, China.","DOI":"10.1145\/3318265.3318276"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hu, G., Jia, Y., and Chen, Z. (2018, January 9\u201313). Multi-user computation offloading with d2d for mobile edge computing. Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/GLOCOM.2018.8647906"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1109\/JIOT.2015.2497712","article-title":"Energy-efficient resource allocation for D2D communications underlaying cloud-RAN-based LTE-A networks","volume":"3","author":"Zhou","year":"2015","journal-title":"IEEE Internet Things J."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1750","DOI":"10.1109\/TWC.2019.2896999","article-title":"D2D communications meet mobile edge computing for enhanced computation capacity in cellular networks","volume":"18","author":"He","year":"2019","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4110","DOI":"10.1109\/JSYST.2019.2921115","article-title":"Task Execution Cost Minimization-Based Joint Computation Offloading and Resource Allocation for Cellular D2D MEC Systems","volume":"13","author":"Chai","year":"2019","journal-title":"IEEE Syst. J."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"6018","DOI":"10.1109\/JIOT.2021.3110319","article-title":"Energy-Efficient Joint Task Assignment and Power Control in Energy-Harvesting D2D Offloading Communications","volume":"9","author":"Hamdi","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3226","DOI":"10.1109\/JIOT.2021.3097754","article-title":"Joint Task Offloading, D2D Pairing, and Resource Allocation in Device-Enhanced MEC: A Potential Game Approach","volume":"9","author":"Fang","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_29","first-page":"1","article-title":"Computation Offloading and Resource Allocation in MEC-Enabled Integrated Aerial-Terrestrial Vehicular Networks: A Reinforcement Learning Approach","volume":"14","author":"Waqar","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"107496","DOI":"10.1016\/j.comnet.2020.107496","article-title":"A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective","volume":"182","author":"Shakarami","year":"2020","journal-title":"Comput. Netw."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1896","DOI":"10.1109\/JIOT.2020.3015970","article-title":"Service-oriented energy-latency tradeoff for IoT task partial offloading in MEC-enhanced multi-RAT networks","volume":"8","author":"Qin","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"9025","DOI":"10.1109\/JIOT.2021.3118016","article-title":"Lyapunov-based Partial Computation Offloading for Multiple Mobile Devices Enabled by Harvested Energy in MEC","volume":"9","author":"Guo","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"13196","DOI":"10.1109\/JIOT.2021.3064995","article-title":"Partial computation offloading in NOMA-assisted mobile-edge computing systems using deep reinforcement learning","volume":"8","author":"Truong","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"16067","DOI":"10.1109\/TVT.2020.3041929","article-title":"Priority-aware task offloading in vehicular fog computing based on deep reinforcement learning","volume":"69","author":"Shi","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"4472","DOI":"10.1109\/TVT.2020.2978027","article-title":"Latency Minimization for D2D-Enabled Partial Computation Offloading in Mobile Edge Computing","volume":"69","author":"Saleem","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"103","DOI":"10.3389\/fnbot.2019.00103","article-title":"Constrained deep q-learning gradually approaching ordinary q-learning","volume":"13","author":"Ohnishi","year":"2019","journal-title":"Front. Neurorobot."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1109\/TWC.2012.121112.120018","article-title":"Joint Resource Allocation and User Association for Heterogeneous Wireless Cellular Networks","volume":"12","author":"Fooladivanda","year":"2013","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Bu, T., Li, L., and Ramjee, R. (2007, January 10). Generalized Proportional Fair Scheduling in Third Generation Wireless Data Networks. Proceedings of the IEEE INFOCOM 2006\u201425th IEEE International Conference on Computer Communications, Barcelona, Spain.","DOI":"10.1109\/INFOCOM.2006.145"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1145\/322077.322090","article-title":"\u201cStrong\u201d np-completeness results: Motivation, examples, and implications","volume":"25","author":"Garey","year":"1978","journal-title":"J. ACM"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/7004\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:32:21Z","timestamp":1760142741000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/7004"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,15]]},"references-count":39,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22187004"],"URL":"https:\/\/doi.org\/10.3390\/s22187004","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,9,15]]}}}