{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T02:51:56Z","timestamp":1761706316432,"version":"build-2065373602"},"reference-count":80,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2014,6,25]],"date-time":"2014-06-25T00:00:00Z","timestamp":1403654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper offers a compacted mechanism to carry out the performance evaluation work for an automatic target recognition (ATR) system: (a) a standard description of the ATR system\u2019s output is suggested, a quantity to indicate the operating condition is presented based on the principle of feature extraction in pattern recognition, and a series of indexes to assess the output in different aspects are developed with the application of statistics; (b) performance of the ATR system is interpreted by a quality factor based on knowledge of engineering mathematics; (c) through a novel utility called \u201ccontext-probability\u201d estimation proposed based on probability, performance prediction for an ATR system is realized. The simulation result shows that the performance of an ATR system can be accounted for and forecasted by the above-mentioned measures. Compared to existing technologies, the novel method can offer more objective performance conclusions for an ATR system. These conclusions may be helpful in knowing the practical capability of the tested ATR system. At the same time, the generalization performance of the proposed method is good.<\/jats:p>","DOI":"10.3390\/s140711308","type":"journal-article","created":{"date-parts":[[2014,6,26]],"date-time":"2014-06-26T02:28:30Z","timestamp":1403749710000},"page":"11308-11350","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["A Compact Methodology to Understand, Evaluate, and Predict the Performance of Automatic Target Recognition"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4037-0001","authenticated-orcid":false,"given":"Yanpeng","family":"Li","sequence":"first","affiliation":[{"name":"School of Electrical Science and Engineering, National University of Defense Technology, 137 Yanwachi Street, Changsha 410073, China"}]},{"given":"Xiang","family":"Li","sequence":"additional","affiliation":[{"name":"School of Electrical Science and Engineering, National University of Defense Technology, 137 Yanwachi Street, Changsha 410073, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2522-9552","authenticated-orcid":false,"given":"Hongqiang","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Electrical Science and Engineering, National University of Defense Technology, 137 Yanwachi Street, Changsha 410073, China"}]},{"given":"Yiping","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Electrical Science and Engineering, National University of Defense Technology, 137 Yanwachi Street, Changsha 410073, China"}]},{"given":"Zhaowen","family":"Zhuang","sequence":"additional","affiliation":[{"name":"School of Electrical Science and Engineering, National University of Defense Technology, 137 Yanwachi Street, Changsha 410073, China"}]},{"given":"Yongqiang","family":"Cheng","sequence":"additional","affiliation":[{"name":"School of Electrical Science and Engineering, National University of Defense Technology, 137 Yanwachi Street, Changsha 410073, China"}]},{"given":"Bin","family":"Deng","sequence":"additional","affiliation":[{"name":"School of Electrical Science and Engineering, National University of Defense Technology, 137 Yanwachi Street, Changsha 410073, China"}]},{"given":"Liandong","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Complex Electromagnetic Environment Effects on Electronics and Information System (CEMEE), Luoyang 471003, China"}]},{"given":"Yonghu","family":"Zeng","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Complex Electromagnetic Environment Effects on Electronics and Information System (CEMEE), Luoyang 471003, China"}]},{"given":"Lei","family":"Gao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Complex Electromagnetic Environment Effects on Electronics and Information System (CEMEE), Luoyang 471003, China"}]}],"member":"1968","published-online":{"date-parts":[[2014,6,25]]},"reference":[{"key":"ref_1","first-page":"3","article-title":"An overview of automatic target recognition","volume":"6","author":"Dudgeon","year":"1993","journal-title":"Linc. Lab. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"8534","DOI":"10.3390\/s130708534","article-title":"Seismic target classification using a wavelet packet manifold in unattended ground sensors systems","volume":"13","author":"Huang","year":"2013","journal-title":"Sensors"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1109\/TAES.1986.310772","article-title":"Automatic target recognition: State of the art survey","volume":"22","author":"Bhanu","year":"1986","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"10181","DOI":"10.3390\/s101110181","article-title":"Extended target recognition in cognitive radar networks","volume":"10","author":"Wei","year":"2010","journal-title":"Sensors"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Tait, P. (2005). Introduction to Radar Target Recognition, The Institution of Electrical Engineers.","DOI":"10.1049\/PBRA018E"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/72.80203","article-title":"Survey of neural network technology for automatic target recognition","volume":"1","author":"Roth","year":"1990","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"8139","DOI":"10.3390\/s8128139","article-title":"Use of automatic target recognition system for the displacement measurements in a small diameter tunnel ahead of the face of the motorway tunnel during excavation","volume":"8","author":"Bogatin","year":"2008","journal-title":"Sensors"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1408","DOI":"10.1126\/science.1716784","article-title":"A combinatorial approach toward DNA recognition","volume":"253","author":"Pei","year":"1991","journal-title":"Science"},{"key":"ref_9","unstructured":"Li, Y. (2004). Performance Evaluation in Automatic Target Recognition: Foundation, Theoretic Framework and Related Research. [Ph.D. Thesis, National University of Defense Technology]."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3400","DOI":"10.1109\/TIP.2006.881965","article-title":"Ground target recognition using rectangle estimation","volume":"15","author":"Gronwall","year":"2006","journal-title":"IEEE Trans. Image Process."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1109\/7.937475","article-title":"Support vector machines for SAR automatic target recognition","volume":"37","author":"Zhao","year":"2001","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Diaz-Bolado, A., Barriere, P.A., and Laurin, J.J. (2013). Study of microwave tomography measurement setup configurations for breast cancer detection based on breast compression. Int. J. Antennas Propag., 2013.","DOI":"10.1155\/2013\/701378"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1126\/science.1195858","article-title":"Poly(A) tail recognition by a viral RNA element through assembly of a triple helix","volume":"330","author":"DeGregorio","year":"2010","journal-title":"Science"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sadjadi, E., and Javid, B. (2007). Physics of Automatic Target Recognition, Springer.","DOI":"10.1007\/978-0-387-36943-3"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1109\/TPAMI.2002.1033209","article-title":"Information-theoretic bounds on target recognition performance based on degraded image data","volume":"24","author":"Jain","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5381","DOI":"10.3390\/s130405381","article-title":"Computational burden resulting from image recognition of high resolution radar sensors","volume":"13","author":"Bravo","year":"2013","journal-title":"Sensors"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"526","DOI":"10.1038\/sj.cgt.7700128","article-title":"Pharmacologic suppression of target cell recognition by engineered T cells expressing chimeric T-cell receptors","volume":"7","author":"Yanez","year":"2000","journal-title":"Cancer Gene Ther."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Nasr, H., and Sadjadi, F. (1989, January 27). Automatic target recognition algorithm performance evaluation: The bottleneck in the development life cycle. Orlando, FL, USA.","DOI":"10.1117\/12.960434"},{"key":"ref_19","unstructured":"Bassham, C.B. (2002). Automatic Target Recognition Classification System Evaluation Methodology. [Ph.D. Thesis, Air Force Institute of Technology, Wright-Patterson AFB]."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ralph, S.K., Irvine, J.M., Snorrason, M., Stevens, M.R., and Vanstone, D. (2005, January 19\u201321). An image metric-based ATR performance prediction testbed. Washington, DC, USA.","DOI":"10.1109\/AIPR.2006.13"},{"key":"ref_21","unstructured":"Fu, Q., and He, J. (2013). The Methodologies and Applications of Performance Evaluation Technology for Automatic Target Recognition, National Defense Industry Press."},{"key":"ref_22","unstructured":"Lui, H.S., and Shuley, N.V. (2010, January 11\u201317). Performance evaluation of subsurface target recognition based on ultra-wide-band short-pulse excitation. Toronto, ON, Canada."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1109\/TSMCC.2009.2020174","article-title":"An objective function to evaluate performance of human-robot collaboration in target recognition tasks","volume":"39","author":"Bechar","year":"2009","journal-title":"IEEE Trans. Syst. Man Cybern. Part C Appl. Rev."},{"key":"ref_24","unstructured":"Irvine, J.M. (2001, January 1\u201312). Evaluation of ATR algorithms employing motion imagery. Washington, DC, USA."},{"key":"ref_25","unstructured":"Zelnio, E.G. (1997, January 28). Extensibility and other model-based ATR evaluation concepts. Orlando, FL, USA."},{"key":"ref_26","unstructured":"Zhuang, Z., Li, X., Li, Y., and Wang, H. (2006). Performance Evaluation Technology for Automatic Target Recognition, National Defense Industry Press."},{"key":"ref_27","unstructured":"Zelnio, E.G. (1998, January 14\u201317). An evaluation of SAR ATR algorithm performance sensitivity to MSTAR extended operating conditions. Orlando, FL, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1109\/62.240102","article-title":"Image understanding research for automatic target recognition","volume":"8","author":"Bhanu","year":"1993","journal-title":"IEEEA erosp. Electron. Syst. Mag."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1117\/12.55784","article-title":"Image characterization for automatic target recognition algorithm evaluations","volume":"30","author":"Clark","year":"1991","journal-title":"Opt. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kaufman, V.I., Ross, T.D., Lavely, E.M., and Blasch, E.P. (2007, January 9). Score-based SAR ATR performance model with operating condition dependencies. Orlando, FL, USA.","DOI":"10.1117\/12.719426"},{"key":"ref_31","unstructured":"Zelnio, E.G., and Garber, F. (1996, January 8\u201310). A characterization of ATR performance evaluation. Orlando, FL, USA."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Nasr, H.N., and Sadjadi, F.A. (1990, January 19\u201320). Automatic evaluation and adaptation of automatic target recognition systems. Orlando, FL, USA.","DOI":"10.1117\/12.21800"},{"key":"ref_33","unstructured":"Wang, E., Li, Y., and Li, X. (2008, January 2\u20135). Performance evaluation for automatic target recognition based on cloud theory. Adelaide, Australia."},{"key":"ref_34","unstructured":"Alsing, S., and Blasch, E.P. (1999, January 13\u201317). Baue, R. 3D ROC surface concepts for evaluation of target recognition algorithms faced with the unknown target detection problem. Orlando, FL, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1006\/dspr.2000.0375","article-title":"ATR.Performance modeling and estimation","volume":"10","author":"Dudgeon","year":"2000","journal-title":"Digit. Signal Process."},{"key":"ref_36","first-page":"49","article-title":"Effects of polarization and resolution on the performance of a SAR automatic target recognition system","volume":"8","author":"Novak","year":"1995","journal-title":"Line. Lab. J."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"7715","DOI":"10.3390\/s8127715","article-title":"Assessment of polarimetric SAR interferometry for improving ship classification based on simulated data","volume":"8","author":"Margarit","year":"2008","journal-title":"Sensors"},{"key":"ref_38","unstructured":"Zelnio, E.G., and Garber, F.D. (2003, January 21). A comparison of SAR ATR performance with information theoretic predictions. Orlando, FL, USA."},{"key":"ref_39","first-page":"11","article-title":"Performance of a high-resolution polarimetric SAR automatic target recognition aystem","volume":"6","author":"Novak","year":"1993","journal-title":"Line. Lab. J."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"17193","DOI":"10.3390\/s131217193","article-title":"Sensor reliability evaluation scheme for target classification using belief function theory","volume":"13","author":"Zhu","year":"2013","journal-title":"Sensors"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1109\/JSEN.2009.2038120","article-title":"Multiparameter receiver operating characteristic analysis for signal detection and classification","volume":"10","author":"Chang","year":"2010","journal-title":"IEEE Sens. J."},{"key":"ref_42","unstructured":"Alsing, S.G. (2000). The Evaluation of Competing Classifiers. [Ph.D. Thesis, Air Force Institute of Technology, Wright-Patterson AFB]."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Parker, D.R., Gustafson, S.C., and Ross, T.D. (2005). Receiver operating characteristic and confidence error metrics for assessing the performance of automatic target recognition systems. Opt. Eng., 44.","DOI":"10.1117\/1.2042307"},{"key":"ref_44","unstructured":"Boshra, M., and Bhanu, B. (1999, January 5). Bounding SAR ATR performance based on model similarity. Orlando, FL, USA."},{"key":"ref_45","unstructured":"Zelnio, E.G. (2001, January 16\u021319). Analytic performance bounds on SAR-image target recognition using physics-based signatures. Orlando, FL, USA."},{"key":"ref_46","unstructured":"Sadjadi, F.A. (2005, January 28). Information theoretic bounds of ATR algorithm performance for sidescan sonar target classifcation. Orlando, FL, USA."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1109\/34.709572","article-title":"Hilbert-schmidt lower bounds for estimators on matrix lie groups for ATR","volume":"20","author":"Grenander","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach Intell."},{"key":"ref_48","unstructured":"Zelnio, E.G. (2001, January 16\u201319). Confidence intervals for ATR performance metrics. Orlando, FL, USA."},{"key":"ref_49","unstructured":"Zelnio, E.G. (2005, January 28\u201331). Fidelity Score for ATR Performance Modeling. Orlando, FL, USA."},{"key":"ref_50","unstructured":"Ren, Q. (2003). Research on Recognition Performance of Automatic Fingerprint Verification Algorithms. [Ph.D. Thesis, Chinese Academy of Sciences]."},{"key":"ref_51","unstructured":"Schmalz, M.S., Ritter, G.X., and Caimi, F.M. (1997, January 6\u20139). Performance evaluation of data compression transforms for underwater imaging and object recognition. Halifax, NS, Canada."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/j.imavis.2011.12.007","article-title":"Demographic effects on estimates of automatic face recognition performance","volume":"30","author":"Phillips","year":"2012","journal-title":"Image Vis. Comput."},{"key":"ref_53","unstructured":"Sadjadi, F.A. (2002, January 1\u20135). Open source tools for ATR development and performance evaluation. Orlando, FL, USA."},{"key":"ref_54","unstructured":"Sadjadi, F.A. (2004, January 12). A scoring, truthing, and registration toolkit for evaluation of target detection and tracking. Orlando, FL, USA."},{"key":"ref_55","unstructured":"Zelnio, E.G., and Garber, F.D. (2009, January 13). Shadow-based SAR ATR performance prediction. Orlando, FL, USA."},{"key":"ref_56","unstructured":"Zelnio, E.G. (1998, January 13). Upper bound calculation of ATR performance for ladar sensors. Orlando, FL, USA."},{"key":"ref_57","unstructured":"Appleby, R., Wikner, D.A., Trebits, R., and Kurtz, J.L. (2003, January 23\u201324). Performance modeling of vote-based object recognition. Orlando, FL, USA."},{"key":"ref_58","unstructured":"Kovalerchuk, B. (2009, January 6\u20139). Modeling ATR processes to predict their performance by using invariance, robustness and self-refusal approach. Seattle, WA, USA."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Chiang, H.C., and Moses, R.L. (1999, January 5\u20139). ATR performance prediction using attributed scattering features. Orlando, FL, USA.","DOI":"10.1117\/12.357693"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Schmalz, M.S. (1997, January 21\u201324). Automated analysis and prediction of accuracy and performance in ATR algorithms: I. Requirements, theory, and software implementation. Orlando, FL, USA.","DOI":"10.1117\/12.280857"},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Schmalz, M.S. (1997, January 21\u201324). Automated analysis and prediction of accuracy and performance in ATR algorithms: II. Experimental results and system performance analysis. Orlando, FL.","DOI":"10.1117\/12.280917"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Horowitz, L.L., and Brendel, G.F. (1999, January 5). Fundamental SAR ATR performance predictions for design tradeoffs: 1-D HRR versus 2-D SAR versus 3-D SAR. Orlando, FL, USA.","DOI":"10.1117\/12.357692"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1016\/j.specom.2009.03.006","article-title":"Computer Assisted Language Learning system based on dynamic question generation and error prediction for automatic speech recognition","volume":"51","author":"Wang","year":"2009","journal-title":"Speech Commun."},{"key":"ref_64","unstructured":"Sadjadi, F.A. (2007, January 9). Evaluation testbed for ATD performance prediction (ETAPP). Orlando, FL, USA."},{"key":"ref_65","unstructured":"(2013, January 26). ATR algorithm performance evaluation based on the simulation image and real image. Wuhan, China."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/34.615449","article-title":"Aided and automatic target recognition based upon sensory inputs from image forming systems","volume":"19","author":"Ratches","year":"1997","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_67","unstructured":"Tou, J.T., and Gonzalez, R.C. (1974). Pattern Recognition Principles, Addison Wesley."},{"key":"ref_68","unstructured":"Ugarte, M.D., Militino, A.F., and Arnholt, A.T. (2009). Probability and Statistics with R, CRC Press."},{"key":"ref_69","unstructured":"Devore, J.L., and Berk, K.N. (2007). Modern Mathematical Statistics with Applications, Thomson Brooks\/Cole."},{"key":"ref_70","unstructured":"Greenberg, M.D. (2011). Advanced Engineering Mathematics, Publishing House of Electronics Industry."},{"key":"ref_71","unstructured":"Leon-Garcia, A. (2009). Probability, Statistics, and Random Processes for Electrical Engineering, Pearson Education Inc."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Cesa-Bianchi, N., and Lugosi, G. (2006). Prediction, Learning, and Games, Cambridge University Press.","DOI":"10.1017\/CBO9780511546921"},{"key":"ref_73","unstructured":"Christopher, D.A., and Hinkley, D.V. (1997). Bootstrap Methods and Their Application, Cambridge University Press."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"3713","DOI":"10.1109\/TGRS.2011.2162526","article-title":"Automatic target recognition of SAR images based on global scattering center model","volume":"49","author":"Zhou","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sen."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1109\/TGRS.2010.2052623","article-title":"An improved scheme for target discrimination in high-resolution SAR images","volume":"49","author":"Gao","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sen."},{"key":"ref_76","first-page":"843","article-title":"The detection and recognition of electrocardiogram's waveform based on sparse decomposition and neural network","volume":"27","author":"Liu","year":"2011","journal-title":"Signal Process."},{"key":"ref_77","first-page":"891","article-title":"EEG signal classification based on EMD and SVM","volume":"28","author":"Li","year":"2011","journal-title":"Biomed. Eng."},{"key":"ref_78","first-page":"61","article-title":"A infrared target recognition method based on classifier combination","volume":"34","author":"Wang","year":"2012","journal-title":"J. Detect. Control."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank PhysioToolkit. PhysioNet: Components of a New Research Resource for Complex Physiologic Signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"ref_80","unstructured":"EEG Database. Available online: http:\/\/archive.ics.uci.edu\/ml\/datasets\/EEG+Database."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/14\/7\/11308\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:12:55Z","timestamp":1760217175000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/14\/7\/11308"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,6,25]]},"references-count":80,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2014,7]]}},"alternative-id":["s140711308"],"URL":"https:\/\/doi.org\/10.3390\/s140711308","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2014,6,25]]}}}