{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,29]],"date-time":"2026-03-29T17:48:08Z","timestamp":1774806488170,"version":"3.50.1"},"reference-count":53,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2023,4,11]],"date-time":"2023-04-11T00:00:00Z","timestamp":1681171200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Technology, Republic of China","award":["MOST 111-2221-E-126-003"],"award-info":[{"award-number":["MOST 111-2221-E-126-003"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The advent of simultaneous wireless information and power (SWIPT) has been regarded as a promising technique to provide power supplies for an energy sustainable Internet of Things (IoT), which is of paramount importance due to the proliferation of high data communication demands of low-power network devices. In such networks, a multi-antenna base station (BS) in each cell can be utilized to concurrently transmit messages and energies to its intended IoT user equipment (IoT-UE) with a single antenna under a common broadcast frequency band, resulting in a multi-cell multi-input single-output (MISO) interference channel (IC). In this work, we aim to find the trade-off between the spectrum efficiency (SE) and energy harvesting (EH) in SWIPT-enabled networks with MISO ICs. For this, we derive a multi-objective optimization (MOO) formulation to obtain the optimal beamforming pattern (BP) and power splitting ratio (PR), and we propose a fractional programming (FP) model to find the solution. To tackle the nonconvexity of FP, an evolutionary algorithm (EA)-aided quadratic transform technique is proposed, which recasts the nonconvex problem as a sequence of convex problems to be solved iteratively. To further reduce the communication overhead and computational complexity, a distributed multi-agent learning-based approach is proposed that requires only partial observations of the channel state information (CSI). In this approach, each BS is equipped with a double deep Q network (DDQN) to determine the BP and PR for its UE with lower computational complexity based on the observations through a limited information exchange process. Finally, with the simulation experiments, we verify the trade-off between SE and EH, and we demonstrate that, apart from the FP algorithm introduced to provide superior solutions, the proposed DDQN algorithm also shows its performance gain in terms of utility to be up to 1.23-, 1.87-, and 3.45-times larger than the Advantage Actor Critic (A2C), greedy, and random algorithms, respectively, in comparison in the simulated environment.<\/jats:p>","DOI":"10.3390\/s23083900","type":"journal-article","created":{"date-parts":[[2023,4,12]],"date-time":"2023-04-12T02:08:11Z","timestamp":1681265291000},"page":"3900","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Joint Data Transmission and Energy Harvesting for MISO Downlink Transmission Coordination in Wireless IoT Networks"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5603-5200","authenticated-orcid":false,"given":"Jain-Shing","family":"Liu","sequence":"first","affiliation":[{"name":"Department of Computer Science and Information Engineering, Providence University, Taichung 433, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0840-394X","authenticated-orcid":false,"given":"Chun-Hung","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, National Sun Yat-sen University, Kaohsiung 804, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5055-3645","authenticated-orcid":false,"given":"Yu-Chen","family":"Hu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Management, Providence University, Taichung 433, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8233-6071","authenticated-orcid":false,"given":"Praveen Kumar","family":"Donta","sequence":"additional","affiliation":[{"name":"Research Unit of Distributed Systems, TU Wien, 1040 Vienna, Austria"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ni, W., Zheng, J., and Tian, H. (2023). Semi-federated learning for collaborative intelligence in massive IoT networks. IEEE Internet Things J.","DOI":"10.1109\/JIOT.2023.3253853"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Wang, W., Chen, J., Jiao, Y., Kang, J., Dai, W., and Xu, Y. (2023). Connectivity-aware contract for incentivizing IoT devices in complex wireless blockchain. IEEE Internet Things J.","DOI":"10.1109\/JIOT.2023.3239928"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1109\/MCOM.2011.5706317","article-title":"Coordinated multipoint: Concepts, performance, and field trial results","volume":"49","author":"Irmer","year":"2011","journal-title":"IEEE Commun. Mag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1109\/49.730452","article-title":"Transmit beamforming and power control for cellular wireless systems","volume":"16","author":"Liu","year":"1998","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_5","unstructured":"(2022, December 22). 3GPP TR36.814. Evolved Universal Terrestrial Radio Access (E-UTRA); Further Advancements for E-UTRA Physical Layer Aspects. Available online: https:\/\/portal.3gpp.org\/desktopmodules\/Specifications\/SpecificationDetails.aspx?specificationId=2493."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"8816","DOI":"10.1109\/JIOT.2021.3050612","article-title":"Massive wireless energy transfer: Enabling sustainable IoT toward 6G era","volume":"8","author":"Alves","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1384","DOI":"10.1109\/COMST.2015.2497324","article-title":"Advances in energy harvesting communications: Past, present, and future challenges","volume":"18","author":"Ku","year":"2016","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/JSAC.2018.2872615","article-title":"Fundamentals of wireless information and power transfer: From RF energy harvester models to signal and system designs","volume":"37","author":"Clerckx","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1989","DOI":"10.1109\/TWC.2013.031813.120224","article-title":"MIMO broadcasting for simultaneous wireless information and power transfer","volume":"12","author":"Zhang","year":"2013","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"6249","DOI":"10.1109\/TSP.2014.2355781","article-title":"Wireless information and energy transfer in multi-antenna interference channel","volume":"62","author":"Shen","year":"2014","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4754","DOI":"10.1109\/TCOMM.2013.13.120855","article-title":"Wireless information and power transfer: Architecture design and rate-energy tradeoff","volume":"61","author":"Zhou","year":"2013","journal-title":"IEEE Trans. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kumar, D., L\u00f3pez, O.L.A., T\u00f6lli, A., and Joshi, S. (2021, January 13\u201316). Latency-aware joint transmit beamforming and receive power splitting for SWIPT systems. Proceedings of the 2021 IEEE 32nd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Helsinki, Finland.","DOI":"10.1109\/PIMRC50174.2021.9569464"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1792","DOI":"10.1109\/TVT.2019.2960760","article-title":"Throughput maximization of a hybrid EH-SWIPT relay system under temperature constraints","volume":"69","author":"Oshaghi","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1109\/JSAC.2014.141212","article-title":"Throughput optimal policies for energy harvesting wireless transmitters with non-ideal circuit power","volume":"32","author":"Xu","year":"2014","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"6352","DOI":"10.1109\/TWC.2013.103113.130470","article-title":"Wireless information and power transfer: Energy efficiency optimization in OFDMA systems","volume":"12","author":"Ng","year":"2013","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1109\/TSP.2015.2489603","article-title":"Energy efficiency optimization for MISO SWIPT systems with zero-forcing beamforming","volume":"64","author":"Shi","year":"2016","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2189","DOI":"10.1109\/LSP.2015.2464082","article-title":"An efficiency maximization design for SWIPT","volume":"22","author":"Vu","year":"2015","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yu, H., Zhang, Y., Guo, S., Yang, Y., and Ji, L. (2017). Energy efficiency maximization for WSNs with simultaneous wireless information and power transfer. Sensors, 17.","DOI":"10.3390\/s17081906"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"7672","DOI":"10.1109\/TWC.2017.2753223","article-title":"Wireless power transfer-based multi-pair two-way relaying with massive antennas","volume":"16","author":"Wang","year":"2017","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"8384","DOI":"10.1109\/JIOT.2020.2990378","article-title":"Wirelessly powered cell-free IoT: Analysis and optimization","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1109\/COMST.2014.2368999","article-title":"Wireless networks with RF energy harvesting: A contemporary survey","volume":"17","author":"Lu","year":"2015","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Huda, S.M.A., Arafat, M.Y., and Moh, S. (2022). Wireless power transfer in wirelessly powered sensor networks: A review of recent progress. Sensors, 22.","DOI":"10.3390\/s22082952"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"8979","DOI":"10.1109\/JIOT.2020.2999892","article-title":"Transmitter-oriented dual-mode SWIPT with deep-learning-based adaptive mode switching for iot sensor networks","volume":"7","author":"Park","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"64495","DOI":"10.1109\/ACCESS.2022.3182686","article-title":"Balancing fairness and energy efficiency in SWIPT-based D2D networks: Deep reinforcement learning based approach","volume":"10","author":"Han","year":"2022","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3130","DOI":"10.1109\/JSYST.2021.3098860","article-title":"Energy efficiency optimization for SWIPT-based D2D-underlaid cellular networks using multiagent deep reinforcement learning","volume":"16","author":"Muy","year":"2022","journal-title":"IEEE Syst. J."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2331","DOI":"10.1109\/TCOMM.2021.3051680","article-title":"Antenna clustering for simultaneous wireless information and power transfer in a MIMO full-duplex system: A deep reinforcement learning-based design","volume":"69","author":"Akrout","year":"2021","journal-title":"IEEE Trans. Commun."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1109\/JSAC.2021.3118397","article-title":"Joint coordinated beamforming and power splitting ratio optimization in MU-MISO SWIPT-enabled hetnets: A multi-agent DDQN-based approach","volume":"40","author":"Zhang","year":"2022","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"8361","DOI":"10.1109\/TVT.2021.3055205","article-title":"Joint optimization of spectral efficiency and energy harvesting in D2D networks using deep neural network","volume":"70","author":"Sengly","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"15039","DOI":"10.1109\/JIOT.2021.3103768","article-title":"Joint subcarrier and transmission power allocation in OFDMA-based WPT system for mobile-edge computing in iot environment","volume":"9","author":"Han","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_31","unstructured":"Han, J., Lee, G.H., Park, S., and Choi, J.K. (2020). Joint orthogonal band and power allocation for energy fairness in WPT system with nonlinear logarithmic energy harvesting model. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2308","DOI":"10.1109\/TWC.2019.2963833","article-title":"Power allocation for D2D communications with SWIPT","volume":"19","author":"Huang","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lu, W., Liu, G., Si, P., Zhang, G., Li, B., and Peng, H. (2019). Joint resource optimization in simultaneous wireless information and power transfer (SWIPT) enabled multi-relay internet of things (IoT) system. Sensors, 19.","DOI":"10.3390\/s19112536"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Lee, K. (2021). Distributed transmit power control for energy-efficient wireless-powered secure communications. Sensors, 21.","DOI":"10.3390\/s21175861"},{"key":"ref_35","unstructured":"Ehrgott, M. (2005). Multicriteria Optimization, Springer Science & Business Media."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"492","DOI":"10.1287\/mnsc.13.7.492","article-title":"On nonlinear fractional programming","volume":"13","author":"Dinkelbach","year":"1967","journal-title":"Manag. Sci."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2616","DOI":"10.1109\/TSP.2018.2812733","article-title":"Fractional programming for communication systems-part I: Power control and beamforming","volume":"66","author":"Shen","year":"2018","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_38","unstructured":"Radaideh, M.I., Du, K., Seurin, P., Seyler, D., Gu, X., Wang, H., and Shirvan, K. (2021). Neorl: Neuroevolution optimization with reinforcement learning. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2239","DOI":"10.1109\/JSAC.2019.2933973","article-title":"Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks","volume":"37","author":"Nasir","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"6070","DOI":"10.1109\/TCOMM.2020.3004524","article-title":"Deep reinforcement learning for distributed dynamic MISO downlink-beamforming coordination","volume":"68","author":"Ge","year":"2020","journal-title":"IEEE Trans. Commun."},{"key":"ref_41","unstructured":"Bertsekas, D.P. (1995). Dynamic Programming and Optimal Control, Athena Scientific."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Tiong, T., Saad, I., Teo, K.T.K., and Lago, H.b. (September, January 31). Deep reinforcement learning with robust deep deterministic policy gradient. Proceedings of the 2020 Second International Conference on Electrical, Control and Instrumentation Engineering (ICECIE), London, UK.","DOI":"10.1109\/ICECIE50279.2020.9309539"},{"key":"ref_43","unstructured":"Fujimoto, S., Hoof, H.V., and Meger, D. (2018, January 13\u201316). Addressing function approximation error in actor-critic methods. Proceedings of the 35th International Conference on Machine Learning, Helsinki, Finland."},{"key":"ref_44","unstructured":"Hasselt, H.V., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double Q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"120604","DOI":"10.1109\/ACCESS.2020.3007002","article-title":"Collaborative edge computing and caching with deep reinforcement learning decision agents","volume":"8","author":"Ren","year":"2020","journal-title":"IEEE Access"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"8918","DOI":"10.1109\/TITS.2021.3087833","article-title":"Delay-aware content delivery with deep reinforcement learning in internet of vehicles","volume":"23","author":"Nan","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zou, W., Cui, Z., Li, B., Zhou, Z., and Hu, Y. (2011, January 12\u201314). Beamforming codebook design and performance evaluation for 60 GHz wireless communication. Proceedings of the 2011 11th International Symposium on Communications and Information Technologies (ISCIT), Hangzhou, China.","DOI":"10.1109\/ISCIT.2011.6089755"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"4589","DOI":"10.1109\/TVT.2014.2374237","article-title":"Learning based frequency- and time-domain inter-cell interference coordination in HetNets","volume":"64","author":"Simsek","year":"2015","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"8577","DOI":"10.1109\/JIOT.2019.2921159","article-title":"Deep deterministic policy gradient (DDPG)-based energy harvesting wireless communications","volume":"6","author":"Qiu","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_50","unstructured":"Bach, F., and Blei, D. (2015, January 7\u20139). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on Machine Learning, Lille, France."},{"key":"ref_51","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing Atari with Deep Reinforcement Learning. arXiv."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Canese, L., Cardarilli, G.C., Nunzio, L.D., Fazzolari, R., Giardino, D., Re, M., and Spano, S. (2021). Multi-agent reinforcement learning: A review of challenges and applications. Appl. Sci., 11.","DOI":"10.3390\/app11114948"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/8\/3900\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:14:13Z","timestamp":1760123653000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/8\/3900"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,11]]},"references-count":53,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23083900"],"URL":"https:\/\/doi.org\/10.3390\/s23083900","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,11]]}}}