{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:23:53Z","timestamp":1760235833726,"version":"build-2065373602"},"reference-count":29,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2021,9,29]],"date-time":"2021-09-29T00:00:00Z","timestamp":1632873600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100012331","name":"Flanders Innovation and Entrepreneurship","doi-asserted-by":"publisher","award":["HBC.2016.0436\/HBC.2018.2028"],"award-info":[{"award-number":["HBC.2016.0436\/HBC.2018.2028"]}],"id":[{"id":"10.13039\/100012331","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Deep neural networks have achieved state-of-the-art performance in image classification. Due to this success, deep learning is now also being applied to other data modalities such as multispectral images, lidar and radar data. However, successfully training a deep neural network requires a large reddataset. Therefore, transitioning to a new sensor modality (e.g., from regular camera images to multispectral camera images) might result in a drop in performance, due to the limited availability of data in the new modality. This might hinder the adoption rate and time to market for new sensor technologies. In this paper, we present an approach to leverage the knowledge of a teacher network, that was trained using the original data modality, to improve the performance of a student network on a new data modality: a technique known in literature as knowledge distillation. By applying knowledge distillation to the problem of sensor transition, we can greatly speed up this process. We validate this approach using a multimodal version of the MNIST dataset. Especially when little data is available in the new modality (i.e., 10 images), training with additional teacher supervision results in increased performance, with the student network scoring a test set accuracy of 0.77, compared to an accuracy of 0.37 for the baseline. We also explore two extensions to the default method of knowledge distillation, which we evaluate on a multimodal version of the CIFAR-10 dataset: an annealing scheme for the hyperparameter \u03b1 and selective knowledge distillation. Of these two, the first yields the best results. Choosing the optimal annealing scheme results in an increase in test set accuracy of 6%. Finally, we apply our method to the real-world use case of skin lesion classification.<\/jats:p>","DOI":"10.3390\/s21196523","type":"journal-article","created":{"date-parts":[[2021,10,8]],"date-time":"2021-10-08T21:26:20Z","timestamp":1633728380000},"page":"6523","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Data-Efficient Sensor Upgrade Path Using Knowledge Distillation"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0165-1337","authenticated-orcid":false,"given":"Pieter","family":"Van Molle","sequence":"first","affiliation":[{"name":"IDLab, Department of Information and Technology, Ghent University, 9052 Gent, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0763-8114","authenticated-orcid":false,"given":"Cedric","family":"De Boom","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information and Technology, Ghent University, 9052 Gent, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2731-7262","authenticated-orcid":false,"given":"Tim","family":"Verbelen","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information and Technology, Ghent University, 9052 Gent, Belgium"}]},{"given":"Bert","family":"Vankeirsbilck","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information and Technology, Ghent University, 9052 Gent, Belgium"}]},{"given":"Jonas","family":"De Vylder","sequence":"additional","affiliation":[{"name":"Barco Healthcare, Barco N.V., 8500 Kortrijk, Belgium"}]},{"given":"Bart","family":"Diricx","sequence":"additional","affiliation":[{"name":"Barco Healthcare, Barco N.V., 8500 Kortrijk, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9569-9373","authenticated-orcid":false,"given":"Pieter","family":"Simoens","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information and Technology, Ghent University, 9052 Gent, Belgium"}]},{"given":"Bart","family":"Dhoedt","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information and Technology, Ghent University, 9052 Gent, Belgium"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,29]]},"reference":[{"key":"ref_1","first-page":"097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_2","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_5","unstructured":"Tan, M., and Le, Q.V. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_7","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Gupta, S., Hoffman, J., and Malik, J. (2016, January 27\u201330). Cross modal distillation for supervision transfer. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.309"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Albanie, S., Nagrani, A., Vedaldi, A., and Zisserman, A. (2018, January 22\u201326). Emotion recognition in speech using cross-modal transfer in the wild. Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Korea.","DOI":"10.1145\/3240508.3240578"},{"key":"ref_10","first-page":"892","article-title":"Soundnet: Learning sound representations from unlabeled video","volume":"29","author":"Aytar","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zisserman, A., Afouras, T., and Chung, J. (2020, January 4\u20138). ASR is all you need: Cross-modal distillation for lip reading. Proceedings of the International Conference of Acoustics, Speech, and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9054253"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Xu, R., Wang, X., Hou, P., Tang, H., and Song, M. (2020, January 7\u201312). Hearing lips: Improving lip reading by distilling speech recognizers. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i04.6174"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ren, S., Du, Y., Lv, J., Han, G., and He, S. (2021, January 19\u201325). Learning From the Master: Distilling Cross-Modal Advanced Knowledge for Lip Reading. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Virtual.","DOI":"10.1109\/CVPR46437.2021.01312"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Thoker, F.M., and Gall, J. (2019, January 22\u201325). Cross-modal knowledge distillation for action recognition. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.","DOI":"10.1109\/ICIP.2019.8802909"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Crasto, N., Weinzaepfel, P., Alahari, K., and Schmid, C. (2019, January 15\u201320). Mars: Motion-augmented rgb stream for action recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00807"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Dai, R., Das, S., and Bremond, F. (2021). Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection. arXiv.","DOI":"10.1109\/ICCV48922.2021.01281"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_18","unstructured":"Krizhevsky, A., and Hinton, G. (2021, September 25). Learning Multiple Layers of Features from Tiny Images. Available online: https:\/\/www.cs.toronto.edu\/~kriz\/learning-features-2009-TR.pdf."},{"key":"ref_19","unstructured":"American Cancer Society (2020). Cancer Facts & Figures 2020, American Cancer Society."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1016\/j.jaad.2017.08.016","article-title":"Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images","volume":"78","author":"Marchetti","year":"2018","journal-title":"J. Am. Acad. Dermatol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"622","DOI":"10.1016\/j.jaad.2019.07.016","article-title":"Computer algorithms show potential for improving dermatologists\u2019 accuracy to diagnose cutaneous melanoma: Results of the International Skin Imaging Collaboration 2017","volume":"82","author":"Marchetti","year":"2020","journal-title":"J. Am. Acad. Dermatol."},{"key":"ref_23","unstructured":"Gutman, D., Codella, N.C., Celebi, E., Helba, B., Marchetti, M., Mishra, N., and Halpern, A. (2016). Skin lesion analysis toward melanoma detection: A challenge at the international symposium on biomedical imaging (ISBI) 2016, hosted by the international skin imaging collaboration (ISIC). arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Codella, N.C., Gutman, D., Celebi, M.E., Helba, B., Marchetti, M.A., Dusza, S.W., Kalloo, A., Liopyris, K., Mishra, N., and Kittler, H. (2018, January 4\u20137). Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA.","DOI":"10.1109\/ISBI.2018.8363547"},{"key":"ref_25","unstructured":"Codella, N., Rotemberg, V., Tschandl, P., Celebi, M.E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., and Marchetti, M. (2019). Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"301","DOI":"10.3322\/caac.20074","article-title":"The evolution of melanoma diagnosis: 25 years beyond the ABCDs","volume":"60","author":"Rigel","year":"2010","journal-title":"CA A Cancer J. Clin."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"708","DOI":"10.1111\/srt.12859","article-title":"Enhanced visualization of blood and pigment in multispectral skin dermoscopy","volume":"26","author":"Janssen","year":"2020","journal-title":"Ski. Res. Technol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/sdata.2018.161","article-title":"The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions","volume":"5","author":"Tschandl","year":"2018","journal-title":"Sci. Data"},{"key":"ref_29","unstructured":"Combalia, M., Codella, N.C., Rotemberg, V., Helba, B., Vilaplana, V., Reiter, O., Carrera, C., Barreiro, A., Halpern, A.C., and Puig, S. (2019). BCN20000: Dermoscopic lesions in the wild. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/19\/6523\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:07:34Z","timestamp":1760166454000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/19\/6523"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,29]]},"references-count":29,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["s21196523"],"URL":"https:\/\/doi.org\/10.3390\/s21196523","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,9,29]]}}}