{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:10:56Z","timestamp":1750219856918,"version":"3.41.0"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T00:00:00Z","timestamp":1680220800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003246","name":"Netherlands Organisation for Scientific Research","doi-asserted-by":"crossref","award":["652-001-001"],"award-info":[{"award-number":["652-001-001"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Comput. Cult. Herit."],"published-print":{"date-parts":[[2023,3,31]]},"abstract":"<jats:p>\n            In recent years,\n            <jats:bold>deep learning (DL)<\/jats:bold>\n            has achieved impressive successes in many application domains, including Handwritten-Text Recognition. However, DL methods demand a long training process and a huge amount of human-based labeled data. To address these issues, we explore several label-free heuristics for detecting the early-stopping point in training convolutional-neural networks: (1) Cumulative Distribution of the\n            <jats:bold>standard deviation of kernel weights (SKW)<\/jats:bold>\n            ; (2) the moving standard deviation of SKW, and (3) the standard deviation of the sum of weights over a window in the epoch series. We applied the proposed methods to the common RIMES and Bentham data sets as well as another highly challenging historical data set. In comparison with the usual stopping criterion which uses labels for validation, the label-free heuristics are at least 10 times faster per epoch when the same training set is used. The use of alternative stopping heuristics may require additional epochs, however, they never require the original computing time. The character error rate (%) on the test set of the label-free heuristics is about a percentage point less in comparison to the usual stopping criterion. The label-free early-stopping methods have two benefits: They do not require a computationally intensive evaluation of a validation set per epoch and all labels can be used for training, specifically benefitting the underrepresented word or letter classes.\n          <\/jats:p>","DOI":"10.1145\/3587168","type":"journal-article","created":{"date-parts":[[2023,4,27]],"date-time":"2023-04-27T11:56:15Z","timestamp":1682596575000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["How to Limit Label Dissipation in Neural-network Validation: Exploring Label-free Early-stopping Heuristics"],"prefix":"10.1145","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9327-2774","authenticated-orcid":false,"given":"Mahya","family":"Ameryan","sequence":"first","affiliation":[{"name":"University of Groningen, The Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2351-930X","authenticated-orcid":false,"given":"Lambert","family":"Schomaker","sequence":"additional","affiliation":[{"name":"University of Groningen, The Netherlands"}]}],"member":"320","published-online":{"date-parts":[[2023,6]]},"reference":[{"key":"e_1_3_4_2_2","series-title":"Proceedings of the International Conference Collect and Connect: Archives and Collections in a Digital Age, Leiden, the Netherlands, November 23-24, 2020","first-page":"92","volume":"2810","author":"Ameryan Mahya","year":"2020","unstructured":"Mahya Ameryan and Lambert Schomaker. 2020. A high-performance word recognition system for the biological fieldnotes of the Natuurkundige Csommissie. In Proceedings of the International Conference Collect and Connect: Archives and Collections in a Digital Age, Leiden, the Netherlands, November 23-24, 2020(CEUR Workshop Proceedings, Vol. 2810), Andreas Weber, Maarten Heerlien, Eul\u00e0lia Gass\u00f3 Miracle, and Katherine Wolstencroft (Eds.). CEUR-WS.org, Leiden, Netherlands, 92\u2013103. http:\/\/ceur-ws.org\/Vol-2810\/paper8.pdf."},{"key":"e_1_3_4_3_2","first-page":"13","volume-title":"17th Int. Conf. Frontiers in Handwriting Recognition","author":"Ameryan Mahya","year":"2020","unstructured":"Mahya Ameryan and Lambert Schomaker. 2020. Improving the robustness of LSTMs for word classification using stressed word endings in dual-state word-beam search. In 17th Int. Conf. Frontiers in Handwriting Recognition. IEEE, Dortmund, Germany, 13\u201318."},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-020-05612-0"},{"key":"e_1_3_4_5_2","series-title":"Proceedings of the 19th International Conference on Artificial Intelligence and Statistics","first-page":"1070","volume":"51","author":"Duvenaud David","year":"2016","unstructured":"David Duvenaud, Dougal Maclaurin, and Ryan Adams. 2016. Early stopping as nonparametric variational inference. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics(Proceedings of Machine Learning Research, Vol. 51), Arthur Gretton and Christian C. Robert (Eds.). PMLR, Cadiz, Spain, 1070\u20131077. https:\/\/proceedings.mlr.press\/v51\/duvenaud16.html."},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/DAS.2014.23"},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.5555\/3086952"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-24797-2"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/1143844.1143891"},{"key":"e_1_3_4_10_2","volume-title":"Econometric Analysis (3rd ed.)","author":"Greene William H.","year":"1997","unstructured":"William H. Greene. 1997. Econometric Analysis (3rd ed.). Macmillan Publishing Company, New York, USA."},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","DOI":"10.30560\/ilr.v1n1p18"},{"key":"e_1_3_4_12_2","first-page":"61","volume-title":"Actes Colloque International Francophone sur l\u2019Ecrit et le Document (CIFED\u201906)","author":"Grosicki Emmanu\u00e8le","year":"2006","unstructured":"Emmanu\u00e8le Grosicki, Matthieu Carr\u00e9, Edouard Geoffrois, Emmanuel Augustin, and Francoise Preteux. 2006. La campagne d\u2019\u00e9valuation RIMES pour la reconnaissance de courriers manuscrits. In Actes Colloque International Francophone sur l\u2019Ecrit et le Document (CIFED\u201906). Fribourg, Switzerland, 61\u201366."},{"key":"e_1_3_4_13_2","first-page":"2958","volume-title":"Proceedings of the International Joint Conference on Neural Networks (IJCNN\u201906)","author":"Guyon Isabelle","year":"2006","unstructured":"Isabelle Guyon, Amir Reza Saffari Azar Alamdari, Gideon Dror, and Joachim M. Buhmann. 2006. Performance prediction challenge. InProceedings of the International Joint Conference on Neural Networks (IJCNN\u201906). IEEE, Vancouver, BC, Canada, 2958\u20132965."},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","DOI":"10.5555\/1865609.1865663"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_4_16_2","unstructured":"Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Optimization. arxiv:1412.6980 [cs.LG]."},{"key":"e_1_3_4_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/0304-4076(92)90104-Y"},{"key":"e_1_3_4_18_2","first-page":"255","volume-title":"The Handbook of Brain Theory and Neural Networks","author":"LeCun Yann","year":"1998","unstructured":"Yann LeCun and Yoshua Bengio. 1998. Convolutional networks for images, speech, and time series. In The Handbook of Brain Theory and Neural Networks, Michael A. Arbib (Ed.). MIT Press, Cambridge, MA, USA, 255\u2013258."},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-2789(96)00139-X"},{"key":"e_1_3_4_20_2","first-page":"630","volume-title":"Generalization and Parameter Estimation in Feedforward Nets: Some Experiments","author":"Morgan Nelson","year":"1990","unstructured":"Nelson Morgan and Herve Bourlard. 1990. Generalization and Parameter Estimation in Feedforward Nets: Some Experiments. Vol. 2. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 630\u2013637."},{"key":"e_1_3_4_21_2","doi-asserted-by":"publisher","DOI":"10.5555\/3104322.3104425"},{"key":"e_1_3_4_22_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-014-0007-7"},{"key":"e_1_3_4_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35289-8_5"},{"key":"e_1_3_4_24_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1969.tb00775.x"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.3390\/app9071345"},{"key":"e_1_3_4_26_2","first-page":"1","volume-title":"International Conference on Learning Representations","author":"Rieck Bastian","year":"2019","unstructured":"Bastian Rieck, Matteo Togninalli, Christian Bock, Michael Moor, Max Horn, Thomas Gumbsch, and Karsten Borgwardt. 2019. Neural persistence: A complexity measure for deep neural networks using algebraic topology. In International Conference on Learning Representations. ICLR2019, New Orleans, Louisiana, USA, 1\u201325. https:\/\/openreview.net\/forum?id=ByxkijC5FQ."},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICFHR-2018.2018.00052"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","DOI":"10.1137\/0711066"},{"key":"e_1_3_4_29_2","first-page":"201","volume-title":"Computational Intelligence: Theories, Applications and Future Directions - Volume II","author":"Susan Seba","year":"2019","unstructured":"Seba Susan, Rohit Ranjan, Udyant Taluja, Udyant Shivang Rai, and Pranav Agarwal. 2019. Neural net optimization by weight-entropy monitoring. In Computational Intelligence: Theories, Applications and Future Directions - Volume II, Nishchal K. Verma and A. K. Ghosh (Eds.). Springer Singapore, Singapore, 201\u2013213."},{"key":"e_1_3_4_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICFHR.2014.137"},{"key":"e_1_3_4_31_2","first-page":"1","article-title":"The shape of learning curves: A review","volume":"2103","author":"Viering Tom","year":"2021","unstructured":"Tom Viering and Marco Loog. 2021. The shape of learning curves: A review. ArXiv abs\/2103.10948 (2021), 1\u201346.","journal-title":"ArXiv"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF02478259"},{"key":"e_1_3_4_33_2","doi-asserted-by":"publisher","DOI":"10.18352\/bmgn-lchr.10741"},{"key":"e_1_3_4_34_2","doi-asserted-by":"crossref","first-page":"80","DOI":"10.4324\/9780429273360-8","volume-title":"The Routledge Handbook of Science and Empire","author":"Weber Andreas","year":"2021","unstructured":"Andreas Weber. 2021. The Routledge Handbook of Science and Empire. Routledge, London and New York, Chapter Natural history collections and empire, 80\u201386."},{"key":"e_1_3_4_35_2","first-page":"155","volume-title":"Final Conference of the Marie Sk\u0142odowska-Curie Initial Training Network for Digital Cultural Heritage, ITN-DCH 2017","volume":"10605","author":"Weber Andreas","year":"2017","unstructured":"Andreas Weber, Mahya Ameryan, Katherine Wolstencroft, Lise Stork, Maarten Heerlien, and Lambert Schomaker. 2017. Towards a digital infrastructure for illustrated handwritten archives. In Final Conference of the Marie Sk\u0142odowska-Curie Initial Training Network for Digital Cultural Heritage, ITN-DCH 2017, M. Ioannides (Ed.), Vol. 10605. Springer, Olimje, Slovenia, 155\u2013166."},{"key":"e_1_3_4_36_2","volume-title":"Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences","author":"Werbos P.","year":"1974","unstructured":"P. Werbos. 1974. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Ph. D. Dissertation. Harvard University."},{"volume-title":"Letter Frequency","year":"2022","key":"e_1_3_4_37_2","unstructured":"Wikipedia. 2022. Letter Frequency. https:\/\/en.wikipedia.org\/wiki\/Letter_frequency."},{"key":"e_1_3_4_38_2","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/1168\/2\/022022"},{"key":"e_1_3_4_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.01.106"},{"key":"e_1_3_4_40_2","volume-title":"The Psycho-biology of Language: An Introduction to Dynamic Philology","author":"Zipf G. K.","year":"1935","unstructured":"G. K. Zipf. 1935. The Psycho-biology of Language: An Introduction to Dynamic Philology. Houghton, Mufflin, Oxford."}],"container-title":["Journal on Computing and Cultural Heritage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3587168","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3587168","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:47:15Z","timestamp":1750178835000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3587168"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,31]]},"references-count":39,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,3,31]]}},"alternative-id":["10.1145\/3587168"],"URL":"https:\/\/doi.org\/10.1145\/3587168","relation":{},"ISSN":["1556-4673","1556-4711"],"issn-type":[{"type":"print","value":"1556-4673"},{"type":"electronic","value":"1556-4711"}],"subject":[],"published":{"date-parts":[[2023,3,31]]},"assertion":[{"value":"2021-10-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-09","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}