{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T18:45:52Z","timestamp":1771613152750,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T00:00:00Z","timestamp":1717632000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>First-order algorithms have long dominated the training of deep neural networks, excelling in tasks like image classification and natural language processing. Now there is a compelling opportunity to explore alternatives that could outperform current state-of-the-art results. From the estimation theory, the Extended Kalman Filter (EKF) arose as a viable alternative and has shown advantages over backpropagation methods. Current computational advances offer the opportunity to review algorithms derived from the EKF, almost excluded from the training of convolutional neural networks. This article revisits an approach of the EKF with decoupling and it brings the Fully Decoupled Extended Kalman Filter (FDEKF) for training convolutional neural networks in image classification tasks. The FDEKF is a second-order algorithm with some advantages over the first-order algorithms, so it can lead to faster convergence and higher accuracy, due to a higher probability of finding the global optimum. In this research, experiments are conducted on well-known datasets that include Fashion, Sports, and Handwritten Digits images. The FDEKF shows faster convergence compared to other algorithms such as the popular Adam optimizer, the sKAdam algorithm, and the reduced extended Kalman filter. Finally, motivated by the finding of the highest accuracy of FDEKF with images of natural scenes, we show its effectiveness in another experiment focused on outdoor terrain recognition.<\/jats:p>","DOI":"10.3390\/a17060243","type":"journal-article","created":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T03:44:35Z","timestamp":1717645475000},"page":"243","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Training of Convolutional Neural Networks for Image Classification with Fully Decoupled Extended Kalman Filter"],"prefix":"10.3390","volume":"17","author":[{"given":"Armando","family":"Gaytan","sequence":"first","affiliation":[{"name":"Unidad Guadalajara, Centro de Investigaci\u00f3n y de Estudios Avanzados del Instituto Polit\u00e9cnico Nacional, Zapopan 45019, Jalisco, Mexico"}]},{"given":"Ofelia","family":"Begovich-Mendoza","sequence":"additional","affiliation":[{"name":"Unidad Guadalajara, Centro de Investigaci\u00f3n y de Estudios Avanzados del Instituto Polit\u00e9cnico Nacional, Zapopan 45019, Jalisco, Mexico"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8803-9502","authenticated-orcid":false,"given":"Nancy","family":"Arana-Daniel","sequence":"additional","affiliation":[{"name":"Centro Universitario de Ciencias Exactas e Ingenier\u00edas, Universidad de Guadalajara, Guadalajara 44430, Jalisco, Mexico"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/MS.2022.3199621","article-title":"AI Engineering: Realizing the potential of AI","volume":"39","author":"Bosch","year":"2022","journal-title":"IEEE Soft."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mukhamediev, R.I., Symagulov, A., Kuchin, Y., Yakunin, K., and Yelis, M. (2021). From classical machine learning to deep neural networks: A simplified scientometric review. Appl. Sci., 11.","DOI":"10.3390\/app11125541"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.gltp.2021.01.004","article-title":"Machine learning and deep learning applications\u2014A vision","volume":"2","author":"Sharma","year":"2021","journal-title":"Glob. Transitions Proc."},{"key":"ref_4","first-page":"3","article-title":"Deep learning applications","volume":"37","author":"Cao","year":"2022","journal-title":"IEEE Intell. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Venkata Krishna, P., and Obaidat, M. (2020). Emerging Research in Data Engineering Systems and Computer Communication, Springer.","DOI":"10.1007\/978-981-15-0135-7"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.inffus.2017.10.006","article-title":"A survey on deep learning for big data","volume":"42","author":"Zhang","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.1162\/neco.2006.18.7.1527","article-title":"A fast learning algorithm for deep belief nets","volume":"18","author":"Hinton","year":"2006","journal-title":"Neural Comput."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Schmidhuber, J. (2015). Deep learning in neural networks: An overview. arXiv.","DOI":"10.1016\/j.neunet.2014.09.003"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1561\/2000000039","article-title":"Deep Learning: Methods and Applications","volume":"7","author":"Deng","year":"2014","journal-title":"Found. Trends Signal Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2352","DOI":"10.1162\/neco_a_00990","article-title":"Deep convolutional neural networks for image classification: A comprehensive review","volume":"29","author":"Rawat","year":"2017","journal-title":"Neural Comput."},{"key":"ref_11","unstructured":"Puskorius, G.V., and Feldkamp, L.A. (1991, January 8\u201312). Decoupled extended Kalman filter training of feedforward layered networks. Proceedings of the IJCNN-91-Seattle International Joint Conference on Neural Networks, Seattle, WA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Haykin, S. (2001). Kalman Filtering and Neural Networks, John Wiley & Sons, Inc.","DOI":"10.1002\/0471221546"},{"key":"ref_13","unstructured":"Touretzky, D. (1988). Advances in Neural Information Processing Systems 1, Morgan Kaufmann Publishers Inc."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Shah, S., and Palmieri, F. (1990, January 17\u201321). MEKA-a fast, local algorithm for training feedforward neural networks. Proceedings of the 1990 IJCNN International Joint Conference on Neural Networks, San Diego, CA, USA.","DOI":"10.1109\/IJCNN.1990.137822"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Haykin, S. (2001). Kalman Filtering and Neural Networks, John Wiley & Sons, Inc.","DOI":"10.1002\/0471221546"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gaytan, A., Begovich, O., and Arana-Daniel, N. (2023, January 25\u201327). Node-Decoupled Extended Kalman Filter versus Adam Optimizer in Approximation of Functions with Multilayer Neural Networks. Proceedings of the 2023 20th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), Mexico City, Mexico.","DOI":"10.1109\/CCE60043.2023.10332895"},{"key":"ref_17","unstructured":"Kingma, D.P., and Ba, J.L. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference for Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"87","DOI":"10.3233\/IDA-200010","article-title":"sKAdam: An improved scalar extension of KAdam for function optimization","volume":"24","author":"Camacho","year":"2020","journal-title":"IEEE Intell. Data Anal."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1016\/j.neunet.2018.09.012","article-title":"Estimation theory and neural networks revisited: REFK and RSVSF as optimization for deep-learning","volume":"108","author":"Ismail","year":"2018","journal-title":"Neural Netw."},{"key":"ref_20","unstructured":"Heimes, F. (1998, January 14). Extended Kalman filter neural network training: Experimental results and algorithm improvements. Proceedings of the SMC\u201998 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics, San Diego, CA, USA."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2541","DOI":"10.1109\/TSP.2021.3071566","article-title":"An efficient and effective second-order training algorithm for LSTM-based adaptive learning","volume":"69","author":"Vural","year":"2021","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1007\/BF00344251","article-title":"Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position","volume":"36","author":"Fukushima","year":"1980","journal-title":"Biol. Cybern."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chen, L., Li, S., Bai, Q., Ya, J., Jiang, S., and Miao, Y. (2021). Review of Image Classification Algorithms Based on Convolutional Neural Networks. Remote Sens., 13.","DOI":"10.3390\/rs13224712"},{"key":"ref_24","unstructured":"Touretzky, D. (1990). Advances in Neural Information Processing Systems 2, Morgan Kaufmann Publishers Inc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/3448250","article-title":"Deep learning for AI","volume":"64","author":"Bengio","year":"2021","journal-title":"Commun. ACM"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Galanis, N.-I., Vafiadis, P., Mirzaev, K.-G., and Papakostas, G.A. (2022). Convolutional Neural Networks: A Roundup and Benchmark of Their Pooling Layer Variants. Algorithms, 15.","DOI":"10.3390\/a15110391"},{"key":"ref_29","unstructured":"Hinton, G. (2022). The forward-forward algorithm: Some preliminary investigations. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Lu, A., and Honarvar Shakibaei Asli, B. (2023). Seismic Image Identification and Detection Based on Tchebichef Moment Invariant. Electronics, 12.","DOI":"10.3390\/electronics12173692"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chen, B., Zhang, L., Chen, H., Liang, K., and Chen, X. (2021). A novel extended Kalman filter with support vector machine based method for the automatic diagnosis and segmentation of brain tumors. Comput. Methods Programs Biomed., 200.","DOI":"10.1016\/j.cmpb.2020.105797"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A new approach to linear filtering and prediction problems","volume":"82","author":"Kalman","year":"1960","journal-title":"Trans. ASME J. Basic Eng."},{"key":"ref_33","unstructured":"Smith, G.L., Schmidt, S.F., and McGee, L.A. (1962). Application of Statistical Filter Theory to the Optimal Estimation of Position and Velocity on Board a Circumlunar Vehicle, Technical Report R-135."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"103966","DOI":"10.1016\/j.dsp.2023.103966","article-title":"Intelligent estimation: A review of theory, applications, and recent advances","volume":"135","author":"Alsadi","year":"2023","journal-title":"Digit. Signal Process."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1109\/34.141559","article-title":"Comparative analysis of backpropagation and the extended Kalman filter for training multilayer perceptrons","volume":"14","author":"Ruck","year":"1992","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"96","DOI":"10.3103\/S1060992X14020088","article-title":"Training neural networks for classification using the extended Kalman filter: A Comparative Study","volume":"23","author":"Chernodub","year":"2014","journal-title":"Opt. Mem. Neural Netw."},{"key":"ref_37","first-page":"100","article-title":"Neural network training using unscented and extended Kalman filter","volume":"1","author":"Pedrino","year":"2017","journal-title":"Robot Autom. Eng. J."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Gomez-Avila, J., Villase\u00f1or, C., Hernandez-Barragan, J., Arana-Daniel, N., Alanis, A.Y., and Lopez-Franco, C. (2020). Neural PD Controller for an Unmanned Aerial Vehicle Trained with Extended Kalman Filter. Algorithms, 13.","DOI":"10.3390\/a13020040"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"4500","DOI":"10.1109\/TNNLS.2019.2955777","article-title":"diffGrad: An optimization method for convolutional neural networks","volume":"31","author":"Dubey","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"268","DOI":"10.5772\/56545","article-title":"Orthophoto Classification for UGV Path Planning using Heterogeneous Computing","volume":"10","author":"Hudjakov","year":"2013","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2807","DOI":"10.1109\/TGRS.2010.2041783","article-title":"Road Extraction From Satellite Images Using Particle Filtering and Extended Kalman Filtering","volume":"48","author":"Movaghati","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/6\/243\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:54:25Z","timestamp":1760108065000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/6\/243"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,6]]},"references-count":42,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["a17060243"],"URL":"https:\/\/doi.org\/10.3390\/a17060243","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,6]]}}}