{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:35:18Z","timestamp":1760150118335,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2023,10,15]],"date-time":"2023-10-15T00:00:00Z","timestamp":1697328000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"KTH Royal Institute of Technology (via KTH Library\u2019s Open Access Policy)"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, a novel feature generator framework is proposed for handwritten digit classification. The proposed framework includes a two-stage cascaded feature generator. The first stage is based on principal component analysis (PCA), which generates projected data on principal components as features. The second one is constructed by a partially trained neural network (PTNN), which uses projected data as inputs and generates hidden layer outputs as features. The features obtained from the PCA and PTNN-based feature generator are tested on the MNIST and USPS datasets designed for handwritten digit sets. Minimum distance classifier (MDC) and support vector machine (SVM) methods are exploited as classifiers for the obtained features in association with this framework. The performance evaluation results show that the proposed framework outperforms the state-of-the-art techniques and achieves accuracies of 99.9815% and 99.9863% on the MNIST and USPS datasets, respectively. The results also show that the proposed framework achieves almost perfect accuracies, even with significantly small training data sizes.<\/jats:p>","DOI":"10.3390\/s23208477","type":"journal-article","created":{"date-parts":[[2023,10,15]],"date-time":"2023-10-15T10:47:32Z","timestamp":1697366852000},"page":"8477","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Two-Stage Feature Generator for Handwritten Digit Classification"],"prefix":"10.3390","volume":"23","author":[{"given":"M.","family":"Gunler Pirim","sequence":"first","affiliation":[{"name":"Vakifbank, 06200 Ankara, Turkey"}]},{"given":"Hakan","family":"Tora","sequence":"additional","affiliation":[{"name":"Department of Avionics, Atilim University, 06830 Ankara, Turkey"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2483-8070","authenticated-orcid":false,"given":"Kasim","family":"Oztoprak","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Konya Food and Agriculture University, 42080 Konya, Turkey"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1723-5741","authenticated-orcid":false,"given":"\u0130smail","family":"Butun","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, KTH Royal Institute of Technology, SE-114 28 Stockholm, Sweden"},{"name":"Department of Computer Engineering, OSTIM Technical University, 06370 Ankara, Turkey"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2876","DOI":"10.1109\/TNNLS.2018.2890334","article-title":"Morphological Convolutional Neural Network Architecture for Digit Recognition","volume":"30","author":"Mellouli","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1007\/BF00332918","article-title":"Auto-Association by Multilayer Perceptions and Singular Value Decomposition","volume":"59","author":"Bourland","year":"1988","journal-title":"Biol. Cybern."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1109\/34.88570","article-title":"Optimized Feature Extraction and the Bayes Decision and Feed-Forward Classifier","volume":"13","author":"Lowe","year":"1991","journal-title":"IEEE Trans. Paterrn Anal. Mach. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1109\/72.572105","article-title":"On Self-Organizing Algorithms and Networks for Class-Separability Features","volume":"8","author":"Chatterjee","year":"1997","journal-title":"IEEE Tran. Neural Netw."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1109\/72.363467","article-title":"Artificial Neural Networks for Feature Extraction and Multivariate Data Projection","volume":"6","author":"Mao","year":"1995","journal-title":"IEEE Tran. Neural Netw."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1109\/72.554193","article-title":"Decision Boundary Feature Extraction for Neural Networks","volume":"8","author":"Lee","year":"1997","journal-title":"IEEE Tran. Neural Netw."},{"key":"ref_7","unstructured":"Theodoridis, S., and Koutroumbas, K. (2006). Pattern Recognition, Academic Press."},{"key":"ref_8","unstructured":"Jarrett, K., Kavukcuoglu, K., Ranzato, M.A., and LeCun, Y. (October, January 29). What is the best multi-stage architecture for object recognition. Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1872","DOI":"10.1109\/TPAMI.2012.230","article-title":"Invariant scattering convolution networks","volume":"35","author":"Bruna","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","first-page":"208","article-title":"Handwritten character recognition using multiresolution technique and Euclidean distance metric","volume":"3","author":"Patel","year":"2012","journal-title":"J. Signal Inf. Process."},{"key":"ref_11","first-page":"57","article-title":"Handwritten character recognition using multiclass SVM classification with hybrid feature extraction","volume":"10","author":"Ayyaz","year":"2012","journal-title":"Pak. J. Eng. Appl. Sci."},{"key":"ref_12","first-page":"193","article-title":"Handwritten English character and digit recognition using multiclass SVM classifier and using structural micro features","volume":"2","author":"Shubhangi","year":"2009","journal-title":"Int. J. Recent Trends Eng."},{"key":"ref_13","unstructured":"Liu, C.L., and Nakagawa, M. (1999, January 20\u201322). Handwritten numeral recognition using neural networks: Improving the accuracy by discriminative training. Proceedings of the Fifth International Conference on Document Analysis and Recognition, Bangalore, India."},{"key":"ref_14","unstructured":"Suen, C.Y., Liu, K., and Strathy, N.W. (1999). Document Analysis Systems: Theory and Practice, Springer."},{"key":"ref_15","unstructured":"Lee, D.S., and Srihari, S.N. (1993, January 25\u201327). Handprinted digit recognition: A comparison of algorithms. Proceedings of the Third International Workshop on Frontiers of Handwriting Recognition, Buffalo, NY, USA."},{"key":"ref_16","unstructured":"Filatov, A., Nikitin, N., Volgunin, A., and Zelinsky, P. (1999). Document Analysis Systems: Theory and Practice, Springer."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Pan, S., Wang, Y., Liu, C., and Ding, X. (2015, January 18\u201322). A discriminative cascade CNN model for offline handwritten digit recognition. Proceedings of the 14th IAPR International Conference on Machine Vision Applications (MVA), Tokyo, Japan.","DOI":"10.1109\/MVA.2015.7153240"},{"key":"ref_18","first-page":"32","article-title":"Handwritten character recognition using Multi scale neural network training technique","volume":"39","author":"Ganapathy","year":"2008","journal-title":"World Acad. Sci. Eng. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Singh, V., and Lal, S. (2014, January 4\u20135). Digit recognition using single layer neural network with PCA. Proceedings of the Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE), Nadi, Fiji.","DOI":"10.1109\/APWCCSE.2014.7053842"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Soman, S.T., Nandigam, A., and Chakravarthy, V.S. (2013, January 15\u201317). An efficient multi classifier system based on convolutional neural network for offline handwritten Telugu character recognition. Proceedings of the National Conference Communications (NCC, 1\u20135), Delhi, India.","DOI":"10.1109\/NCC.2013.6488008"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5017","DOI":"10.1109\/TIP.2015.2475625","article-title":"PCANet: A simple deep learning baseline for image classification","volume":"24","author":"Chan","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ciresan, D., Meier, U., and Schmidhuber, J. (2012, January 16\u201321). Multi-column deep neural networks for image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248110"},{"key":"ref_23","unstructured":"Goodfellow, I.J., Farley, D.W., Mirza, M., Courville, A., and Bengio, Y. (2013, January 16\u201321). Maxout networks. Proceedings of the 30th ICML, Atlanta, GA, USA."},{"key":"ref_24","unstructured":"Zeiler, M.D., and Fergus, R. (2013, January 2\u20134). Stochastic pooling for regularization of deep convolutional neural networks. Proceedings of the ICLR, Scottsdale, AZ, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Deng, L., and Yu, D. (2011, January 27\u201331). Deep convex network: A scalable architecture for speech pattern classification. Proceedings of the International Speech Communication Association, Florence, Italy.","DOI":"10.21437\/Interspeech.2011-607"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yu, K., Lin, Y., and Lafferty, J. (2011, January 20\u201325). Learning image representations from the pixel level via hierarchical sparse coding. Proceedings of the IEEE Conference CVPR, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995732"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1422","DOI":"10.1109\/TPAMI.2007.1153","article-title":"Deformation models for image recognition","volume":"29","author":"Keysers","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Lee, H., Grosse, R., Rananth, R., and Ng, A.Y. (2009, January 14\u201318). Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. Proceedings of the 26th Annual ICML, Montreal, QC, Canada.","DOI":"10.1145\/1553374.1553453"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hotta, S., Kiyasu, S., and Miyahara, S. (2004, January 26). Pattern recognition using average patterns of categorical k-nearest neighbors. Proceedings of the 17th International Conference on Pattern Recognition (ICPR), Cambridge, UK.","DOI":"10.1109\/ICPR.2004.1333790"},{"key":"ref_30","unstructured":"Mairal, J., Bach, F., Ponce, J., Sapiro, G., and Zisserman, A. (2008, January 8\u201311). Supervised dictionary learning. Proceedings of the Advances in Neural Information Processing Systems NIPS, Vancouver, BC, Canada."},{"key":"ref_31","unstructured":"Zhang, H., Berg, A.C., Maire, M., and Malik, J. (2006, January 17\u201322). SVM-KNN: Discriminative nearest neighbor classification for visual category recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201906), New York, NY, USA."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Su, T.H., Liu, C.L., and Zhang, X.Y. (2011, January 18\u201321). Perceptron Learning of Modified Quadratic Discriminant Function. Proceedings of the International Conference on Document Analysis and Recognition, Beijing, China.","DOI":"10.1109\/ICDAR.2011.204"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1016\/j.patcog.2018.12.023","article-title":"Sparse, collaborative, or nonnegative representation: Which helps pattern classification","volume":"88","author":"Xu","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1007\/s11045-016-0466-4","article-title":"Novel features and a cascaded classifier based Arabic numerals recognition system","volume":"29","author":"Prasad","year":"2018","journal-title":"Multidimens. Syst. Signal Process."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Li, Z., Yang, Z., Yuan, B., and Liu, X. (2023). Air-GR: An Over-the-Air Handwritten Character Recognition System Based on Coordinate Correction YOLOv5 Algorithm and LGR-CNN. Sensors, 23.","DOI":"10.3390\/s23031464"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chen, M., Lin, J., Zou, Y., and Wu, K. (2022). Acoustic Sensing Based on Online Handwritten Signature Verification. Sensors, 22.","DOI":"10.3390\/s22239343"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Campos, C., Sandak, J., Kljun, M., and \u010copi\u010d Pucihar, K. (2022). The Hybrid Stylus: A Multi-Surface Active Stylus for Interacting with and Handwriting on Paper, Tabletop Display or Both. Sensors, 22.","DOI":"10.3390\/s22187058"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Alemayoh, T.T., Shintani, M., Lee, J.H., and Okamoto, S. (2022). Deep-Learning-Based Character Recognition from Handwriting Motion Data Captured Using IMU and Force Sensors. Sensors, 22.","DOI":"10.3390\/s22207840"},{"key":"ref_39","unstructured":"Pirim, M.A.G. (2017). Neural Network Based Feature Extraction for Handwritten Digit Recognition. [Ph.D. Thesis, Atilim University]."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Hou, Y., and Zhao, H. (2017, January 24\u201326). Handwritten Digit Recognition Based on Depth Neural Network. Proceedings of the International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), Okinawa, Japan.","DOI":"10.1109\/ICIIBMS.2017.8279710"},{"key":"ref_41","unstructured":"Bettilyon, T.E. (2022, May 25). How to Classify MNIST Digits with Different Neural Network Architectures. Available online: https:\/\/medium.com\/tebs-lab\/how-to-classify-mnist-digits-with-different-neural-network-architectures-39c75a0f03e3."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1109\/34.291440","article-title":"A database for Handwritten Text Recognition Research","volume":"16","author":"Hull","year":"1994","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Tax, D.M.J., and Laskov, P. (2003, January 17\u201319). Online SVM learning from classification to data description and back. Proceedings of the IEEE 13th Workshop on Neural Networks for Signal Processing (IEEE Cat. No. 03TH8718), Toulouse, France.","DOI":"10.1109\/NNSP.2003.1318049"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/20\/8477\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:07:11Z","timestamp":1760130431000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/20\/8477"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,15]]},"references-count":44,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["s23208477"],"URL":"https:\/\/doi.org\/10.3390\/s23208477","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,10,15]]}}}