{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,26]],"date-time":"2026-04-26T05:21:30Z","timestamp":1777180890280,"version":"3.51.4"},"reference-count":35,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2020,3,2]],"date-time":"2020-03-02T00:00:00Z","timestamp":1583107200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61702551"],"award-info":[{"award-number":["61702551"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>At present, the method of two-dimensional image recognition is mainly used to detect the abnormal fastener in the rail-track inspection system. However, the too-tight-or-too-loose fastener condition may cause the clip of the fastener to break or loose due to the high frequency vibration shock, which is difficult to detect from the two-dimensional image. In this practical application background, 3D visual detection technology provides a feasible solution. In this paper, we propose a fundamental multi-source visual data detection method, as well as an accurate and robust fastener location and nut or bolt segmentation algorithm. By combining two-dimensional intensity information and three-dimensional depth information generated by the projection of line structural light, the locating of nut or bolt position and accurate perception of height information can be realized in the dynamic running environment of railway. The experimental results show that the static measurement accuracy in the vertical direction using the structural light vision sensor is 0.1 mm under the laboratory condition, and the dynamic measurement accuracy is 0.5 mm under the dynamic train running environment. We use dynamic template matching algorithm to locate fasteners from 2D intensity map, which achieves 99.4% accuracy, then use the watershed algorithm to segment the nut and bolt from the corresponding depth image of located fastener. Finally, the 3D shape of the nut and bolt is analyzed to determine whether the nut or bolt height meets the local statistical threshold requirements, so as to detect the hidden danger of railway transportation caused by too loose or too tight fasteners.<\/jats:p>","DOI":"10.3390\/s20051367","type":"journal-article","created":{"date-parts":[[2020,3,3]],"date-time":"2020-03-03T13:06:23Z","timestamp":1583240783000},"page":"1367","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["A Rail Fastener Tightness Detection Approach Using Multi-source Visual Sensor"],"prefix":"10.3390","volume":"20","author":[{"given":"Qiang","family":"Han","sequence":"first","affiliation":[{"name":"School of Science, Beijing Jiaotong University, Beijing 100044, China"}]},{"given":"Shengchun","family":"Wang","sequence":"additional","affiliation":[{"name":"Infrastructure Inspection Research Institute, China Academy of Railway Sciences Corporation Limited, Beijing 100081, China"}]},{"given":"Yue","family":"Fang","sequence":"additional","affiliation":[{"name":"Infrastructure Inspection Research Institute, China Academy of Railway Sciences Corporation Limited, Beijing 100081, China"}]},{"given":"Le","family":"Wang","sequence":"additional","affiliation":[{"name":"Infrastructure Inspection Research Institute, China Academy of Railway Sciences Corporation Limited, Beijing 100081, China"}]},{"given":"Xinyu","family":"Du","sequence":"additional","affiliation":[{"name":"Infrastructure Inspection Research Institute, China Academy of Railway Sciences Corporation Limited, Beijing 100081, China"}]},{"given":"Hailang","family":"Li","sequence":"additional","affiliation":[{"name":"Infrastructure Inspection Research Institute, China Academy of Railway Sciences Corporation Limited, Beijing 100081, China"}]},{"given":"QiXin","family":"He","sequence":"additional","affiliation":[{"name":"School of Science, Beijing Jiaotong University, Beijing 100044, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2854-044X","authenticated-orcid":false,"given":"Qibo","family":"Feng","sequence":"additional","affiliation":[{"name":"School of Science, Beijing Jiaotong University, Beijing 100044, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1109\/LSP.2018.2825947","article-title":"High-Speed railway fastener detection based on a line local binary pattern","volume":"25","author":"Fan","year":"2018","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1101","DOI":"10.1109\/TSMC.2014.2388435","article-title":"Railway fastener inspection by real-time machine vision","volume":"45","author":"Aytekin","year":"2015","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_3","first-page":"33","article-title":"RAILCHECK system: Automated optoelectronic inspection of rail systems","volume":"49","author":"Maiwald","year":"1998","journal-title":"Eisenbahningenieur"},{"key":"ref_4","first-page":"47","article-title":"Application of German RailCheck photoelectric automatic rail detection system in rail inspection vehicle","volume":"4","author":"Zhang","year":"2001","journal-title":"Harbin Railw. Sci. Technol."},{"key":"ref_5","first-page":"144","article-title":"Research and manufacture of vehicle-mounted track inspection system","volume":"11","author":"Han","year":"2014","journal-title":"Railw. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Singh, M., Singh, S., and Jaiswail, L. (2006, January 16\u201317). Autonomous rail track inspection using vision-based system. Proceedings of the 2006 International Conference on Computational Intelligence for Homeland Security and Personal Safety, Alexandria, VA, USA.","DOI":"10.1109\/CIHSPS.2006.313313"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Sholl, H., Ammar, R., and Greenshields, I. (2006, January 24\u201326). Application of computing analysis to real-time railroad track inspection. Proceedings of the 2006 World Automation Congress, Budapest, Hungary.","DOI":"10.1109\/WAC.2006.376027"},{"key":"ref_8","unstructured":"Cai, X.S., Zhu, S.T., and Ma, H. (2005, January 13\u201315). Research on Qinghai-Tibet railway plateau inspection vehicle. Proceedings of the Qinghai-Tibet Railway Operation Management and Related Technology Symposium, Beijing, China."},{"key":"ref_9","first-page":"25","article-title":"Research on robust fast algorithm of rail surface defect detection","volume":"1","author":"Ren","year":"2011","journal-title":"China Railw. Sci."},{"key":"ref_10","first-page":"139","article-title":"Development of the on-board track inspection system based on computer vision","volume":"1","author":"Xu","year":"2013","journal-title":"China Railw. Sci."},{"key":"ref_11","first-page":"267","article-title":"Research on key technology of high-speed and on-board acquisition of digital image","volume":"1","author":"Dai","year":"2014","journal-title":"Comput. Meas. Control"},{"key":"ref_12","unstructured":"Trosino, M., Cunningham, J.J., and Shaw, A.E. (2000). Automated Track Inspection Vehicle and Method. (6,064,428), U.S. Patent."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"877","DOI":"10.1109\/TIM.2013.2283741","article-title":"Automatic fastener classification and defect detection in vision-based railway inspection systems","volume":"63","author":"Hao","year":"2014","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Xia, Y.Q., Xie, F.Y., and Jiang, Z.G. (2010, January 11\u201312). Broken railway fastener detection based on adaboost algorithm. Proceedings of the 2010 International Conference on Optoelectronics and Image Processing, Haikou, China.","DOI":"10.1109\/ICOIP.2010.303"},{"key":"ref_15","first-page":"145","article-title":"Railway fastener locating algorithm based on mutual information","volume":"2","author":"Xie","year":"2013","journal-title":"Chin. J. Stereol. Image Anal."},{"key":"ref_16","unstructured":"Li, Y.B. (2013). Research of the Detection Algorithm on Railway Status Based on Image. [Master\u2019s Thesis, Southwest Jiaotong University]."},{"key":"ref_17","first-page":"110","article-title":"Railway fastener state detection based on HOG feature","volume":"10","author":"Li","year":"2013","journal-title":"Transducer Microsyst. Technol."},{"key":"ref_18","unstructured":"Hang, Y.Y. (2014). Research of Railway Fastener Detection Algorithm Based on Computer Vision. [Master\u2019s Thesis, Southwest Jiaotong University]."},{"key":"ref_19","first-page":"30","article-title":"Optimization of RBF-SVM model in railway fastener detection system","volume":"15","author":"Liu","year":"2015","journal-title":"Comput. Eng. Appl."},{"key":"ref_20","first-page":"1","article-title":"An improved correction algorithm for railway fastener image","volume":"9","author":"Dong","year":"2014","journal-title":"Railw. Comput. Appl."},{"key":"ref_21","unstructured":"Dou, Y.G. (2014). An Adaptive GPGPU-Based bolt Detection System. [Master\u2019s Thesis, Beijing Jiaotong University]."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1007\/s13042-013-0223-z","article-title":"A fast template matching-based algorithm for railway bolts detection","volume":"5","author":"Dou","year":"2014","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Faghih-Roohi, S., Hajizadeh, S., N\u00fa\u00f1ez, A., Babuska, R., and De Schutter, B. (2016, January 24\u201329). Deep convolutional neural networks for detection of rail surface defects. Proceedings of the 2016 International Joint Conference on Neural Networks, Vancouver, BC, Canada.","DOI":"10.1109\/IJCNN.2016.7727522"},{"key":"ref_24","unstructured":"Liu, W.Q. (2016). Railway Obstacle Detection Algorithm Based on Deep Neural Network. [Master\u2019s Thesis, Beijing Jiaotong University]."},{"key":"ref_25","unstructured":"Zhao, X.X. (2016). Railway Fastener Detection Based on Convolution Neural Network. [Master\u2019s Thesis, Beijing Jiaotong University]."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1109\/TITS.2016.2568758","article-title":"Deep multitask learning for railway track inspection","volume":"18","author":"Gibert","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_27","first-page":"164","article-title":"Rail corrosion forensics using 3D imaging and finite element analysis","volume":"3","author":"Safa","year":"2015","journal-title":"Int. J. R. Transp."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"8664","DOI":"10.3390\/s150408664","article-title":"Multi-camera and structured-light vision system (MSVS) for dynamic high-accuracy 3D measurements of railway tunnels","volume":"15","author":"Zhan","year":"2015","journal-title":"Sensors"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhou, Y.H., Wang, S.H., Mei, X., Yin, W.L., Lin, C.F., Hu, Q.W., and Mao, Q.Z. (2017). Railway tunnel clearance inspection method based on 3D point cloud from mobile laser scanning. Sensors, 17.","DOI":"10.3390\/s17092055"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Rikhotso, V., Steyn, N., and Hamam, Y. (2017, January 18\u201320). 3D rail modelling and measurement for rail profile condition assessment. Proceedings of the 2017 IEEE AFRICON, Cape Town, South Africa.","DOI":"10.1109\/AFRCON.2017.8095708"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gabara, G., and Sawicki, P. (2018). A new approach for inspection of selected geometric parameters of a railway track using image-based point clouds. Sensors, 18.","DOI":"10.3390\/s18030791"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Xiong, Z., Li, Q., Mao, Q., and Zou, Q. (2017). A 3D laser profiling system for rail surface defect detection. Sensors, 17.","DOI":"10.3390\/s17081791"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"125201","DOI":"10.1088\/1361-6501\/aa8691","article-title":"Novel method for rail wear inspection based on the sparse iterative closest point method","volume":"28","author":"Yi","year":"2017","journal-title":"Meas. Sci. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/j.isprsjprs.2017.11.007","article-title":"A rigorous fastener inspection approach for high-speed railway from structured light sensors","volume":"143","author":"Mao","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Cui, H., Hu, Q., and Mao, Q. (2018). Real-time geometric parameter measurement of high-speed railway fastener based on point cloud from structured light sensors. Sensors, 18.","DOI":"10.3390\/s18113675"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1367\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:03:25Z","timestamp":1760173405000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1367"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,2]]},"references-count":35,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["s20051367"],"URL":"https:\/\/doi.org\/10.3390\/s20051367","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,2]]}}}