{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T09:41:41Z","timestamp":1772876501119,"version":"3.50.1"},"reference-count":30,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2019,12,18]],"date-time":"2019-12-18T00:00:00Z","timestamp":1576627200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["16SV7867"],"award-info":[{"award-number":["16SV7867"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Meal assistant robots form a very important part of the assistive robotics sector since self-feeding is a priority activity of daily living (ADL) for people suffering from physical disabilities like tetraplegia. A quick survey of the current trends in this domain reveals that, while tremendous progress has been made in the development of assistive robots for the feeding of solid foods, the task of feeding liquids from a cup remains largely underdeveloped. Therefore, this paper describes an assistive robot that focuses specifically on the feeding of liquids from a cup using tactile feedback through force sensors with direct human\u2013robot interaction (HRI). The main focus of this paper is the application of reinforcement learning (RL) to learn what the best robotic actions are, based on the force applied by the user. A model of the application environment is developed based on the Markov decision process and a software training procedure is designed for quick development and testing. Five of the commonly used RL algorithms are investigated, with the intention of finding the best fit for training, and the system is tested in an experimental study. The preliminary results show a high degree of acceptance by the participants. Feedback from the users indicates that the assistive robot functions intuitively and effectively.<\/jats:p>","DOI":"10.3390\/robotics9010001","type":"journal-article","created":{"date-parts":[[2019,12,20]],"date-time":"2019-12-20T09:50:33Z","timestamp":1576835433000},"page":"1","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Application of Reinforcement Learning to a Robotic Drinking Assistant"],"prefix":"10.3390","volume":"9","author":[{"given":"Tejas","family":"Kumar Shastha","sequence":"first","affiliation":[{"name":"Friedrich Wilhelm Bessel Institut Forschungsgesellschaft m.b.H., 28359 Bremen, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0968-2477","authenticated-orcid":false,"given":"Maria","family":"Kyrarini","sequence":"additional","affiliation":[{"name":"Friedrich Wilhelm Bessel Institut Forschungsgesellschaft m.b.H., 28359 Bremen, Germany"}]},{"given":"Axel","family":"Gr\u00e4ser","sequence":"additional","affiliation":[{"name":"Friedrich Wilhelm Bessel Institut Forschungsgesellschaft m.b.H., 28359 Bremen, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2019,12,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ritschel, H., Seiderer, A., Janowksi, K., Wagner, S., and Andre, E. (2019, January 5\u20137). Adaptive linguistic style for an assistive robotic health companion based on explicit human feedback. Proceedings of the 12th ACM International Conference on Pervasive Technologies Related to Assistive Environments, Rhodes, Greece.","DOI":"10.1145\/3316782.3316791"},{"key":"ref_2","unstructured":"Ritschel, H. (2018, January 10\u201315). Socially-aware reinforcement learning for personalized human-robot interaction. Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, Stockholm, Sweden."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Tsiakas, K., Dagioglou, M., Karkaletsis, V., and Makedon, F. (2016, January 1\u20133). Adaptive robot assisted therapy using interactive reinforcement learning. Proceedings of the International Conference on Social Robotics, Kansas City, MO, USA.","DOI":"10.1007\/978-3-319-47437-3_2"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Naotunna, I., Perera, C.J., Sandaruwan, C., Gopura, R.A.R.C., and Lalitharatne, T.D. (2015, January 12\u201313). Meal assistance robots: A review on current status, challenges and future directions. Proceedings of the 2015 IEEE\/SICE International Symposium on System Integration (SII), Nagoya, Japan.","DOI":"10.1109\/SII.2015.7404980"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1080\/10400435.2017.1396565","article-title":"Acceptance and perceived usefulness of robots to assist with activities of daily living and healthcare tasks","volume":"31","author":"Hall","year":"2019","journal-title":"Assist. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1179\/2045772313Y.0000000132","article-title":"Functional assessment and performance evaluation for assistive robotic manipulators: Literature review","volume":"36","author":"Chung","year":"2013","journal-title":"J. Spinal Cord Med."},{"key":"ref_7","unstructured":"Neater Solutions Ltd. (2019, March 08). 2017 Brochure. Available online: http:\/\/www.neater.co.uk\/neater-eater\/."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhang, X., Wang, X., Wang, B., and Sugi, T. (2008, January 14\u201317). Real-time control strategy for EMG-drive meal assistance robot\u2014My Spoon. Proceedings of the 2008 International Conference on Control, Automation and Systems, Seoul, Korea.","DOI":"10.1109\/ICCAS.2008.4694607"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ohshima, Y., Kobayashi, Y., and Kaneko, T. (2013, January 15\u201317). Meal support system with spoon using laser range finder and manipulator. Proceedings of the 2013 IEEE Workshop on Robot Vision (WORV), Clearwater Beach, FL, USA.","DOI":"10.1109\/WORV.2013.6521918"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"95","DOI":"10.3233\/TAD-1999-10203","article-title":"The development of handy 1. A robotic system to assist the severely disabled","volume":"10","author":"Topping","year":"1999","journal-title":"Technol. Disabil."},{"key":"ref_11","unstructured":"B\u00fchler, C., Heck, H., Topping, M., and Smith, J. (1999, January 1\u20134). Practical experiences using the \u2018Handy 1\u2019assistive robot far various ADL tasks. Proceedings of the 5th European Conference for the Advancement of Assistive Technology AAATE 99, D\u00fcsseldorf, Germany."},{"key":"ref_12","unstructured":"Yamamoto, M., Sakai, Y., Funakoshi, Y., and Ishimatsu, T. (1999, January 12\u201315). Assistive robot hand for the disabled. Proceedings of the International Conference on Systems, Man, and Cybernetics (Cat. No. 99CH37028), Tokyo, Japan."},{"key":"ref_13","unstructured":"Lopes, P., Lavoie, R., Faldu, R., Aquino, N., Barron, J., Kante, M., and Magfory, B. (2018, September 01). Icraft-Eye-Controlled Robotic Feeding Arm Technology. Available online: https:\/\/ece.northeastern.edu\/personal\/meleis\/icraft.pdf."},{"key":"ref_14","unstructured":"Mealtime Partner Inc. (2019, March 08). Hands-Free eating and Drinking Products for Individuals with Disabilities. Available online: http:\/\/www.mealtimepartners.com\/."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Schr\u00f6er, S., Killmann, I., Frank, B., Voelker, M., Fiederer, L., Ball, T., and Burgard, W. (2015, January 25\u201330). An autonomous robotic assistant for drinking. Proceedings of the International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7140110"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1023\/B:AURO.0000016864.12513.77","article-title":"Integration of a rehabilitation robotic system (KARES II) with human-friendly man-machine interaction units","volume":"16","author":"Bien","year":"2004","journal-title":"Auton. Robot."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Goldau, F.F., Shastha, T.K., Kyrarini, M., and Gr\u00e4ser, A. (2019, January 24\u201328). Autonomous multi-sensory robotic assistant for a drinking task. Proceedings of the 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR), Toronto, ON, Canada.","DOI":"10.1109\/ICORR.2019.8779521"},{"key":"ref_18","unstructured":"Kinova (2019, February 25). Jaco User Guide. Available online: https:\/\/www.robotshop.com\/media\/files\/PDF\/jaco-arm-user-guide-jaco-academique.pdf."},{"key":"ref_19","unstructured":"(2018, September 01). Tekscan, Flexiforce Datasheet. Available online: https:\/\/www.tekscan.com\/products-solutions\/electronics\/flexiforce-quickstart-board."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Koller, T.L., Kyrarini, M., and Gr\u00e4ser, A. (2019, January 5\u20137). Towards robotic drinking assistance: Low cost multi-sensor system to limit forces in human-robot-interaction. Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments, Rhodes, Greece.","DOI":"10.1145\/3316782.3321539"},{"key":"ref_21","unstructured":"(2019, February 26). National Cancer Institute, Anatomical Terminology, Available online: https:\/\/training.seer.cancer.gov\/anatomy\/body\/terminology.html."},{"key":"ref_22","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_23","unstructured":"Alpaydin, E. (2014). Introduction to Machine Learning, MIT Press."},{"key":"ref_24","unstructured":"Bertsekas, D.P. (2011). Dynamic Programming and Optimal Control, MIT Press. [3rd ed.]."},{"key":"ref_25","unstructured":"Rummery, G.A., and Niranjan, M. (1994). On-Line Q-Learning Using Connectionist Systems, University of Cambridge, Department of Engineering."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_27","unstructured":"(2019, March 03). Keras Documentation, Layers, Core Layers. Available online: https:\/\/keras.io\/layers\/core\/."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: An overview","volume":"61","author":"Schmidhuber","year":"2015","journal-title":"Neural Netw."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: A brief survey","volume":"34","author":"Arulkumaran","year":"2017","journal-title":"IEEE Signal. Process. Mag."},{"key":"ref_30","unstructured":"Brooke, J. (2019, November 15). System Usability Scale (SUS), Available online: https:\/\/www.usability.gov\/how-to-and-tools\/methods\/system-usability-scale.html."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/9\/1\/1\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:43:11Z","timestamp":1760190191000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/9\/1\/1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,18]]},"references-count":30,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["robotics9010001"],"URL":"https:\/\/doi.org\/10.3390\/robotics9010001","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,18]]}}}