{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T04:32:03Z","timestamp":1777696323897,"version":"3.51.4"},"reference-count":28,"publisher":"SAGE Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IDA"],"published-print":{"date-parts":[[2023,1,30]]},"abstract":"<jats:p>In recent years, various loss functions have been proposed to boost the performance of deep neural networks. Every loss function has its own specific theoretical motivation, and can easily learn its preference features of training data compared with other loss functions. Thus, combining multiple loss functions to capture more data features becomes an attractive idea for model performance improvement. In this paper, instead of using a single loss function or a linear weighted sum of multiple loss functions, we present the method named Multiple Independent Losses Scheduling (MILS), which allows multiple loss functions to independently participate in the training process according to their performance. Specifically, for all candidate loss functions, one loss function will be predefined as the primary loss function before training, and the other loss functions will play auxiliary roles for possible contributions to improve the model performance. In order to avoid auxiliary loss functions bringing a negative effect on the model performance in the training process, we developed a simple but effective performance-based scheduling algorithm to prevent auxiliary loss functions from dragging down the model performance. Extensive experiments using various deep architectures on various recognition benchmarks demonstrate our scheme is simple, robust, lightweight, and effective for typical classification tasks.<\/jats:p>","DOI":"10.3233\/ida-216401","type":"journal-article","created":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T11:19:22Z","timestamp":1675163962000},"page":"165-180","source":"Crossref","is-referenced-by-count":0,"title":["Multiple independent losses scheduling: A simple training method for deep neural networks"],"prefix":"10.1177","volume":"27","author":[{"given":"Jiali","family":"Deng","sequence":"first","affiliation":[{"name":"Yangtze Delta Region Institution (Quzhou), University of Electronic Science and Technology of China, Quzhou, Zhejiang, China"}]},{"given":"Haigang","family":"Gong","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China"}]},{"given":"Xiaomin","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China"}]},{"given":"Minghui","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China"},{"name":"Yangtze Delta Region Institution (Quzhou), University of Electronic Science and Technology of China, Quzhou, Zhejiang, China"}]},{"given":"Tianshu","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China"}]},{"given":"Xuan","family":"Cheng","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China"},{"name":"Yangtze Delta Region Institution (Quzhou), University of Electronic Science and Technology of China, Quzhou, Zhejiang, China"}]},{"given":"Ming","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China"}]},{"given":"Wanqing","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China"}]}],"member":"179","reference":[{"issue":"4","key":"10.3233\/IDA-216401_ref1","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1162\/neco.1989.1.4.541","article-title":"Backpropagation applied to handwritten zip code recognition","volume":"1","author":"LeCun","year":"1989","journal-title":"Neural Computation"},{"key":"10.3233\/IDA-216401_ref2","unstructured":"A. Krizhevsky, I. Sutskever and G.E. Hinton, ImageNet Classification with Deep Convolutional Neural Networks, in: Proceedings of the 26th Annual Conference on Neural Information Processing Systems, F. Pereira, C.J.C. Burges, L. Bottou et al., eds, Curran Associates, Inc., 2012, pp.\u00a01106\u20131114. https:\/\/proceedings.neurips.cc\/paper\/2012\/hash\/c399862d3b9d6b76c8436e924a68c45b-Abstract.html."},{"issue":"6","key":"10.3233\/IDA-216401_ref3","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","article-title":"Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups","volume":"29","author":"Hinton","year":"2012","journal-title":"IEEE Signal Processing Magazine"},{"issue":"1","key":"10.3233\/IDA-216401_ref4","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1109\/TASL.2011.2134090","article-title":"Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition","volume":"20","author":"Dahl","year":"2012","journal-title":"IEEE Transactions on Audio, Speech, and Language Processing"},{"key":"10.3233\/IDA-216401_ref5","unstructured":"S. Ioffe and C. Szegedy, Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, in: Proceedings of the 32nd International Conference on Machine Learning, F. Bach and D. Blei, eds, Proceedings of Machine Learning Research, PMLR, 2015, pp.\u00a0448\u2013456. http:\/\/proceedings.mlr.press\/v37\/ioffe15.html."},{"key":"10.3233\/IDA-216401_ref6","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"issue":"6","key":"10.3233\/IDA-216401_ref7","doi-asserted-by":"publisher","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"10.3233\/IDA-216401_ref8","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00093"},{"key":"10.3233\/IDA-216401_ref9","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00783"},{"key":"10.3233\/IDA-216401_ref10","unstructured":"W. Liu, Y. Wen, Z. Yu et al., Large-Margin Softmax Loss for Convolutional Neural Networks, in: Proceedings of the 33nd International Conference on Machine Learning, M.F. Balcan and K.Q. Weinberger, eds, Proceedings of Machine Learning Research, Vol.\u00a048, PMLR, 2016, pp.\u00a0507\u2013516. http:\/\/proceedings.mlr.press\/v48\/liud16.html."},{"key":"10.3233\/IDA-216401_ref11","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46478-7_31"},{"key":"10.3233\/IDA-216401_ref12","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.145"},{"key":"10.3233\/IDA-216401_ref13","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.324"},{"key":"10.3233\/IDA-216401_ref14","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.713"},{"key":"10.3233\/IDA-216401_ref15","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00950"},{"issue":"12","key":"10.3233\/IDA-216401_ref16","doi-asserted-by":"publisher","first-page":"2273","DOI":"10.1109\/TCSVT.2015.2477937","article-title":"Multi-loss regularized deep neural network","volume":"26","author":"Xu","year":"2016","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"10.3233\/IDA-216401_ref17","doi-asserted-by":"crossref","unstructured":"P. Liang, B. Taskar and D. Klein, Alignment by Agreement, in: Proceedings of the Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, Main Conference, R.C. Moore, J.A. Bilmes, J. Chu-Carroll et al., eds, Association for Computational Linguistics, New York City, USA, 2006, pp.\u00a0104\u2013111. https:\/\/www.aclweb.org\/anthology\/N06-1014\/.","DOI":"10.3115\/1220835.1220849"},{"key":"10.3233\/IDA-216401_ref18","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00552"},{"key":"10.3233\/IDA-216401_ref19","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00482"},{"key":"10.3233\/IDA-216401_ref21","unstructured":"Y. Zhang, J.D. Lee and M.I. Jordan, L1-regularized Neural Networks are Improperly Learnable in Polynomial Time, in: Proceedings of the 33nd International Conference on Machine Learning, M.F. Balcan and K.Q. Weinberger, eds, Proceedings of Machine Learning Research, Vol.\u00a048, PMLR, 2016, pp.\u00a0993\u20131001. http:\/\/proceedings.mlr.press\/v48\/zhangd16.html."},{"key":"10.3233\/IDA-216401_ref22","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_32"},{"key":"10.3233\/IDA-216401_ref23","doi-asserted-by":"crossref","unstructured":"W. Tang, G. Hua and L. Wang, How to Train a Compact Binary Neural Network with High Accuracy?, in: Proceedings of the 31st AAAI Conference on Artificial Intelligence, 2017, pp.\u00a02625\u20132631. http:\/\/aaai.org\/ocs\/index.php\/AAAI\/AAAI17\/paper\/view\/14619.","DOI":"10.1609\/aaai.v31i1.10862"},{"key":"10.3233\/IDA-216401_ref24","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00152"},{"key":"10.3233\/IDA-216401_ref25","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.49"},{"key":"10.3233\/IDA-216401_ref26","doi-asserted-by":"crossref","unstructured":"P. Chen, Y. Sharma, H.n. Zhang et al., EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples, in: Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018, pp.\u00a010\u201317. https:\/\/www.aaai.org\/ocs\/index.php\/AAAI\/AAAI18\/paper\/view\/16893.","DOI":"10.1609\/aaai.v32i1.11302"},{"key":"10.3233\/IDA-216401_ref29","unstructured":"K. Simonyan and A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, in: Proceedings of the 3rd International Conference on Learning Representations, Y. Bengio and Y. LeCun, eds, 2015. http:\/\/arxiv.org\/abs\/1409.1556."},{"key":"10.3233\/IDA-216401_ref30","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"10.3233\/IDA-216401_ref31","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.243"}],"container-title":["Intelligent Data Analysis"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/IDA-216401","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:19:50Z","timestamp":1777454390000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/IDA-216401"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,30]]},"references-count":28,"journal-issue":{"issue":"1"},"URL":"https:\/\/doi.org\/10.3233\/ida-216401","relation":{},"ISSN":["1088-467X","1571-4128"],"issn-type":[{"value":"1088-467X","type":"print"},{"value":"1571-4128","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,30]]}}}