{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,18]],"date-time":"2025-12-18T14:27:15Z","timestamp":1766068035825,"version":"build-2065373602"},"reference-count":67,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The museum system is exposed to a high risk of seismic hazards. However, it is difficult to carry out seismic hazard prevention to protect cultural relics in collections due to the lack of real data and diverse types of seismic hazards. To address this problem, we developed a deep-learning-based multi-source feature-fusion method to assess the data on seismic damage caused by collected cultural relics. Firstly, a multi-source data-processing strategy was developed according to the needs of seismic impact analysis of the cultural relics in the collection, and a seismic event-ontology model of cultural relics was constructed. Additionally, a seismic damage data-classification acquisition method and empirical calculation model were designed. Secondly, we proposed a deep learning-based multi-source feature-fusion matching method for cultural relics. By constructing a damage state assessment model of cultural relics using superpixel map convolutional fusion and an automatic data-matching model, the quality and processing efficiency of seismic damage data of the cultural relics in the collection were improved. Finally, we formed a dataset oriented to the seismic damage risk analysis of the cultural relics in the collection. The experimental results show that the accuracy of this method reaches 93.6%, and the accuracy of cultural relics label matching is as high as 82.6% compared with many kinds of earthquake damage state assessment models. This method can provide more accurate and efficient data support, along with a scientific basis for subsequent research on the impact analysis of seismic damage to cultural relics in collections.<\/jats:p>","DOI":"10.3390\/s24144525","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T16:23:41Z","timestamp":1720801421000},"page":"4525","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Multi-Source Feature-Fusion Method for the Seismic Data of Cultural Relics Based on Deep Learning"],"prefix":"10.3390","volume":"24","author":[{"given":"Lin","family":"He","sequence":"first","affiliation":[{"name":"Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Quan","family":"Wei","sequence":"additional","affiliation":[{"name":"Sichuan Museum, Chengdu 610000, China"}]},{"given":"Mengting","family":"Gong","sequence":"additional","affiliation":[{"name":"Sichuan Museum, Chengdu 610000, China"}]},{"given":"Xiaofei","family":"Yang","sequence":"additional","affiliation":[{"name":"Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China"}]},{"given":"Jianming","family":"Wei","sequence":"additional","affiliation":[{"name":"Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,12]]},"reference":[{"key":"ref_1","first-page":"139","article-title":"Seismic protection of cultural relics in museums (3)-Seismic safety analysis of cultural relics on display","volume":"S1","author":"Wu","year":"2002","journal-title":"Prot. Cult. Relics Archaeol. Sci."},{"key":"ref_2","first-page":"39","article-title":"Collection of cultural relics anti-seismic technology in the practice of Ya\u2019 an City Museum","volume":"67","author":"Wu","year":"2022","journal-title":"China Cult. Relics Sci. Res."},{"key":"ref_3","first-page":"13","article-title":"Research on the whole system solutions and measures of earthquake prevention for museums","volume":"S1","author":"Ge","year":"2016","journal-title":"Southeast Cult."},{"key":"ref_4","unstructured":"Pan, J. (2017). Research on Anti-Seismic Protection of Cultural Relics on Display in Museums in Yunnan Province. [Master\u2019s Thesis, Yunnan University]."},{"key":"ref_5","first-page":"9670191","article-title":"Museum Relic Image Detection and Recognition Based on Deep Learning","volume":"2022","author":"Wang","year":"2022","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_6","first-page":"2409","article-title":"Small cultural relics photogrammetry 3D modeling subject object recognition","volume":"25","author":"Niu","year":"2021","journal-title":"J. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"108982","DOI":"10.1016\/j.compeleceng.2023.108982","article-title":"Extraction and analysis algorithms for Sanxingdui cultural relics based on hyperspectral imaging","volume":"111","author":"Qiu","year":"2023","journal-title":"Comput. Electr. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"122399","DOI":"10.1016\/j.eswa.2023.122399","article-title":"Recognizing materials in cultural relic images using computer vision and attention mechanism","volume":"239","author":"Pei","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_9","first-page":"152","article-title":"Research on seismic protection method of cultural relics on display in the Palace Museum","volume":"186","author":"Zhou","year":"2016","journal-title":"J. Palace Mus."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ni, J., Chen, J., Wu, Y., Chen, Z., and Liang, M. (2022). Method to Determine the Centroid of Non-Homogeneous Polygons Based on Suspension Theory. ISPRS Int. J. Geo-Inf., 11.","DOI":"10.3390\/ijgi11040233"},{"key":"ref_11","unstructured":"Yuan, H.Y. (2019). Research and Application of Highly Simulation Technology of Cultural Relics. [Ph.D. Thesis, North University of Technology]."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1109\/TPAMI.2023.3323376","article-title":"Dataset Distillation: A Comprehensive Review","volume":"46","author":"Yu","year":"2024","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"91588","DOI":"10.1109\/ACCESS.2022.3202639","article-title":"FormulaNet: A Benchmark Dataset for Mathematical Formula Detection","volume":"10","author":"Huang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4163","DOI":"10.1109\/TMM.2022.3171688","article-title":"RGBT Salient Object Detection: A Large-Scale Dataset and Benchmark","volume":"25","author":"Tu","year":"2023","journal-title":"IEEE Trans. Multimed."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.patrec.2023.09.015","article-title":"The Florence Multi-Resolution 3D Facial Expression Dataset","volume":"175","author":"Ferrari","year":"2023","journal-title":"Pattern Recognit. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"34083","DOI":"10.1007\/s11042-023-16673-3","article-title":"SAWIT: A Small-Sized Animal Wild Image Dataset with Annotations","volume":"83","author":"Nguyen","year":"2023","journal-title":"Multimed. Tools Appl."},{"key":"ref_17","unstructured":"Feng, Q. Research on the Construction Technology of Knowledge Graph of Cultural Relics. Master\u2019s thesis, Northwest University, Xi\u2019an, China, 2023; pp. 1\u201315."},{"key":"ref_18","first-page":"22","article-title":"Study on seismic damage characteristics and laws of ancient timber frame buildings","volume":"42","author":"Yuan","year":"2022","journal-title":"Earthq. Eng. Eng. Vib."},{"key":"ref_19","first-page":"10","article-title":"Research progress of integrated seismic protection technology for cultural relics in Chinese museums","volume":"1","author":"Ge","year":"2021","journal-title":"China Mus."},{"key":"ref_20","first-page":"79","article-title":"Practice and Reflection on the Digital Protection of Cultural Relics in Shanxi Museum","volume":"06","author":"Qin","year":"2023","journal-title":"China Mus."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Tsiachta, A., Argyrou, P., Tsougas, I., Kladou, M., Ravanidis, P., Kaimaris, D., Georgiadis, C., Georgoula, O., and Patias, P. (2024). Multi-Sensor Image and Range-Based Techniques for the Geometric Documentation and the Photorealistic 3D Modeling of Complex Architectural Monuments. Sensors, 24.","DOI":"10.3390\/s24092671"},{"key":"ref_22","first-page":"37","article-title":"Application of laser technology in cultural relics protection","volume":"59","author":"Wang","year":"2022","journal-title":"Prog. Laser Optoelectron."},{"key":"ref_23","unstructured":"Zheng, Y.Y., Wang, H.Q., and Wang, K. (2024). An improved grid holography projection transformation method for multi-lens spectral image mosaic of cultural relics. Comput. Eng. Appl., 1\u201313."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Fernandez-Balbuena, A.A., G\u00f3mez-Manzanares, A., Mart\u00ednez Ant\u00f3n, J.C., G\u00f3mez-Tejedor, J.G., Mayorga-Pinilla, S., Dur\u00e1n Roque, H., and V\u00e1zquez Molin\u00ed, D. (2023). High-Dynamic-Range Spectral Reflectance for the Segmentation of Paint Pigment: Application to Dal\u00ed\u2019s Oil Painting Dos Figuras (1926). Sensors, 23.","DOI":"10.3390\/s23094316"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"108324","DOI":"10.1016\/j.radphyschem.2019.05.014","article-title":"Cultural Heritage Science at CNA (Seville, Spain): Applications of XRF and IBA Techniques to Art and Archaeological Objects","volume":"167","author":"Ager","year":"2020","journal-title":"Radiat. Phys. Chem."},{"key":"ref_26","unstructured":"Zhang, Q.Y., Zou, Y.L., and Li, J.H. (2023). Application of 3D laser scanning technology in erosion and spalling monitoring of natural cultural relics. Not. Surv. Mapp., 140\u2013144."},{"key":"ref_27","first-page":"87","article-title":"Study on the characteristics of Bronze Cultural relics by X-ray flaw Detection Imaging\u2014A case study of Bronze Cultural relics in the Museum of Huangdao District of Qingdao","volume":"4","author":"Xu","year":"2022","journal-title":"Sci. Res. Cult. Relics China"},{"key":"ref_28","first-page":"126","article-title":"Museum collection knowledge organization for artificial intelligence-Taking the Palace Museum\u2019s \u201cReference Model of Ancient Chinese Movable Cultural Relics\u201d as an example","volume":"11","author":"Zhuang","year":"2023","journal-title":"Palace Mus. J."},{"key":"ref_29","first-page":"51","article-title":"The application of X-ray micro-CT imaging technology in the research of metal cultural relics production process and corrosion state evaluation","volume":"34","author":"Song","year":"2022","journal-title":"Cult. Relics Prot. Archaeol. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.culher.2023.05.015","article-title":"The Visualization Method of Environmental Risk Assessment for Metal Cultural Relics in Museums Based on Quick Response Code","volume":"62","author":"Li","year":"2023","journal-title":"J. Cult. Herit."},{"key":"ref_31","unstructured":"Groves, R., and Liang, H. (2023, January 26\u201327). Multi-Scale Convolutional Autoencoder Network-Based Implicit Information Mining of Cultural Relics. Proceedings of the Optics for Arts, Architecture, and Archaeology (O3A) IX, Munich, Germany."},{"key":"ref_32","first-page":"90","article-title":"Stability analysis of stone monuments under earthquake","volume":"6","author":"Qin","year":"2021","journal-title":"Wen Bo"},{"key":"ref_33","first-page":"85","article-title":"Dynamic characteristics and seismic response analysis of Yuhua Pavilion in the Forbidden City","volume":"36","author":"Zhao","year":"2020","journal-title":"World Earthq. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1558","DOI":"10.4028\/www.scientific.net\/AMM.351-352.1558","article-title":"Seismic damage causes and strengthening methods of museum free-standing cultural relics","volume":"351","author":"Ji","year":"2013","journal-title":"Appl. Mech. Mater."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, W., Li, J., Jiang, J., Wang, B., Wang, Q., Gao, E., and Yue, T. (2024). Autonomous Data Association and Intelligent Information Discovery Based on Multimodal Fusion Technology. Symmetry, 16.","DOI":"10.3390\/sym16010081"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"108908","DOI":"10.1016\/j.ast.2024.108908","article-title":"MHA-Net: Multi-Source Heterogeneous Aerodynamic Data Fusion Neural Network Embedding Reduced-Dimension Features","volume":"145","author":"Ning","year":"2024","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.comcom.2023.07.026","article-title":"A Multimodal Dual-Fusion Entity Extraction Model for Large and Complex Devices","volume":"210","author":"Tong","year":"2023","journal-title":"Comput. Commun."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"109795","DOI":"10.1016\/j.ress.2023.109795","article-title":"Deep Feature Interactive Network for Machinery Fault Diagnosis Using Multi-Source Heterogeneous Data","volume":"242","author":"Miao","year":"2024","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yan, C., Fang, X., Huang, X., Guo, C., and Wu, J. (2023). A Solution and Practice for Combining Multi-Source Heterogeneous Data to Construct Enterprise Knowledge Graph. Front. Big Data, 6.","DOI":"10.3389\/fdata.2023.1278153"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2455001","DOI":"10.1142\/S0218001424550012","article-title":"Research on Multi-Source Heterogeneous Big Data Fusion Method Based on Feature Level","volume":"38","author":"Chen","year":"2024","journal-title":"Int. J. Patt. Recogn. Artif. Intell."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Xu, C., Shu, J., and Zhu, G. (2023). Scene Classification Based on Heterogeneous Features of Multi-Source Data. Remote Sens., 15.","DOI":"10.3390\/rs15020325"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1016\/j.ymeth.2023.08.006","article-title":"Cross-View Contrastive Representation Learning Approach to Predicting DTIs via Integrating Multi-Source Information","volume":"218","author":"He","year":"2023","journal-title":"Methods"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"109397","DOI":"10.1016\/j.ijepes.2023.109397","article-title":"An Integrated Risk Assessment Model for the Multi-Perspective Vulnerability of Distribution Networks under Multi-Source Heterogeneous Data Distributions","volume":"153","author":"Huang","year":"2023","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"35281","DOI":"10.1109\/ACCESS.2023.3264707","article-title":"Analysis of Substation Joint Safety Control System and Model Based on Multi-Source Heterogeneous Data Fusion","volume":"11","author":"Wu","year":"2023","journal-title":"IEEE Access"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"He, X., Dong, H., Yang, W., and Li, W. (2023). Multi-Source Information Fusion Technology and Its Application in Smart Distribution Power System. Sustainability, 15.","DOI":"10.3390\/su15076170"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Jin, Q., Xie, J., Huang, D., Zhao, C., and He, H. (2024). MSFF-MA-DDI: Multi-Source Feature Fusion with Multiple Attention Blocks for Predicting Drug\u2013Drug Interaction Events. Comput. Biol. Chem., 108.","DOI":"10.1016\/j.compbiolchem.2023.108001"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2207","DOI":"10.1007\/s11760-023-02889-1","article-title":"Fabric Image Retrieval Based on Multi-Modal Feature Fusion","volume":"18","author":"Zhang","year":"2024","journal-title":"SIViP"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"112600","DOI":"10.1016\/j.matdes.2023.112600","article-title":"Quantification of Micro-Damage Evolution Process in Ceramics through Extensive Analysis of Multi-Source Heterogeneous Data","volume":"237","author":"Zhang","year":"2024","journal-title":"Mater. Des."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.inffus.2023.01.015","article-title":"Multi-Level Feature Fusion for Multimodal Human Activity Recognition in Internet of Healthcare Things","volume":"94","author":"Islam","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1016\/j.cherd.2023.09.007","article-title":"Fault Diagnosis Strategy of Industrial Process Based on Multi-Source Heterogeneous Information and Deep Learning","volume":"198","author":"Tian","year":"2023","journal-title":"Chem. Eng. Res. Des."},{"key":"ref_51","first-page":"2133","article-title":"Urban Flow Pattern Mining Based on Multi-Source Heterogeneous Data Fusion and Knowledge Graph Embedding","volume":"35","author":"Liu","year":"2023","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"111530","DOI":"10.1016\/j.jece.2023.111530","article-title":"Multi-Source and Multimodal Data Fusion for Improved Management of a Wastewater Treatment Plant","volume":"11","author":"Strelet","year":"2023","journal-title":"J. Environ. Chem. Eng."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"112979","DOI":"10.1016\/j.measurement.2023.112979","article-title":"Real-Time Comprehensive Health Status Assessment of Hydropower Units Based on Multi-Source Heterogeneous Uncertainty Information","volume":"216","author":"Liu","year":"2023","journal-title":"Measurement"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"18694","DOI":"10.1109\/ACCESS.2024.3358453","article-title":"Traffic Accident Risk Prediction of Tunnel Based on Multi-Source Heterogeneous Data Fusion","volume":"12","author":"Wang","year":"2024","journal-title":"IEEE Access"},{"key":"ref_55","unstructured":"(2024, July 02). National Cultural Heritage Administration, Available online: http:\/\/www.ncha.gov.cn\/col\/col2423\/index.html."},{"key":"ref_56","unstructured":"(2024, June 20). Find Open Datasets and Machine Learning Projects|Kaggle. Available online: https:\/\/www.kaggle.com\/datasets."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The Pascal Visual Object Classes (VOC) Challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., and Belongie, S. (2014, January 6\u201312). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Liu, H., Tian, Y., and Wang, Y. (2016, January 27\u201330). Deep relative distance learning: Tell the difference between similar vehicles. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.238"},{"key":"ref_60","unstructured":"Li, W., Zhao, R., and Wang, X. (2012, January 5\u20139). Human Reidentification with Transferred Metric Learning. Proceedings of the Asian Conference on Computer Vision (ACCV), Daejeon, Republic of Korea."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"4513","DOI":"10.1109\/TCSVT.2021.3128054","article-title":"Multi-label image classification via category prototype compositional learning","volume":"32","author":"Zhou","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_62","unstructured":"Chen, T.S., Xu, M.X., and Hui, X.L. (November, January 27). Learning semantic\u2014Specific graph representation for multi-label image recognition. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"3427","DOI":"10.1007\/s13042-023-01841-6","article-title":"Multi-layered semantic representation network for multi-label image classification","volume":"14","author":"Qu","year":"2023","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_64","unstructured":"Zhou, F.T., Huang, S., and Xing, Y. (2021, January 19\u201327). Deep semantic dictionary learning for multi-label image classification. Proceedings of the AAA Conference on Artificial Intelligence, Montreal, QC, Canada."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1007\/s00530-024-01269-0","article-title":"Same-clothes person re-identification with dual-stream network","volume":"30","author":"Wu","year":"2024","journal-title":"Multimed. Syst."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"2991","DOI":"10.1109\/TITS.2020.3027578","article-title":"Dual Domain Multi-Task Model for Vehicle Re-Identification","volume":"23","author":"Huang","year":"2022","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"ref_67","first-page":"488","article-title":"Multi-Attention-Based Soft Partition Network for Vehicle Re-Identification","volume":"10","author":"Lee","year":"2023","journal-title":"J. Comput. Des. Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/14\/4525\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:16:05Z","timestamp":1760109365000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/14\/4525"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,12]]},"references-count":67,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["s24144525"],"URL":"https:\/\/doi.org\/10.3390\/s24144525","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2024,7,12]]}}}