{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T07:58:27Z","timestamp":1768982307251,"version":"3.49.0"},"reference-count":45,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,4,21]],"date-time":"2023-04-21T00:00:00Z","timestamp":1682035200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000780","name":"European-Union-funded Project PolicyCLOUD","doi-asserted-by":"publisher","award":["870675"],"award-info":[{"award-number":["870675"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>The increasing amount of legal information available online is overwhelming for both citizens and legal professionals, making it difficult and time-consuming to find relevant information and keep up with the latest legal developments. Automatic text summarization techniques can be highly beneficial as they save time, reduce costs, and lessen the cognitive load of legal professionals. However, applying these techniques to legal documents poses several challenges due to the complexity of legal documents and the lack of needed resources, especially in linguistically under-resourced languages, such as the Greek language. In this paper, we address automatic summarization of Greek legal documents. A major challenge in this area is the lack of suitable datasets in the Greek language. In response, we developed a new metadata-rich dataset consisting of selected judgments from the Supreme Civil and Criminal Court of Greece, alongside their reference summaries and category tags, tailored for the purpose of automated legal document summarization. We also adopted several state-of-the-art methods for abstractive and extractive summarization and conducted a comprehensive evaluation of the methods using both human and automatic metrics. Our results: (i) revealed that, while extractive methods exhibit average performance, abstractive methods generate moderately fluent and coherent text, but they tend to receive low scores in relevance and consistency metrics; (ii) indicated the need for metrics that capture better a legal document summary\u2019s coherence, relevance, and consistency; (iii) demonstrated that fine-tuning BERT models on a specific upstream task can significantly improve the model\u2019s performance.<\/jats:p>","DOI":"10.3390\/info14040250","type":"journal-article","created":{"date-parts":[[2023,4,21]],"date-time":"2023-04-21T06:54:19Z","timestamp":1682060059000},"page":"250","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Evaluation of Automatic Legal Text Summarization Techniques for Greek Case Law"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0679-5516","authenticated-orcid":false,"given":"Marios","family":"Koniaris","sequence":"first","affiliation":[{"name":"Division of Computer Science, School of Electrical and Computer Engineering, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780 Athens, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-2809-2730","authenticated-orcid":false,"given":"Dimitris","family":"Galanis","sequence":"additional","affiliation":[{"name":"Division of Computer Science, School of Electrical and Computer Engineering, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780 Athens, Greece"}]},{"given":"Eugenia","family":"Giannini","sequence":"additional","affiliation":[{"name":"Department of Humanities Social Sciences and Law, School of Applied Mathematical and Physical Sciences, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780 Athens, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8503-5784","authenticated-orcid":false,"given":"Panayiotis","family":"Tsanakas","sequence":"additional","affiliation":[{"name":"Division of Computer Science, School of Electrical and Computer Engineering, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780 Athens, Greece"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1561\/1500000015","article-title":"Automatic Summarization","volume":"5","author":"Nenkova","year":"2011","journal-title":"Found. Trends\u00ae Inf. Retr."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"100388","DOI":"10.1016\/j.cosrev.2021.100388","article-title":"Summarization of legal documents: Where are we now and the way forward","volume":"40","author":"Jain","year":"2021","journal-title":"Comput. Sci. Rev."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Solan, L.M. (1993). The Language of Judges, University of Chicago Press.","DOI":"10.7208\/chicago\/9780226767895.001.0001"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/BF00877694","article-title":"Text retrieval in the legal world","volume":"3","author":"Turtle","year":"1995","journal-title":"Artif. Intell. Law"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1794","DOI":"10.1016\/j.ipm.2019.04.001","article-title":"A survey on evaluation of summarization methods","volume":"56","author":"Ermakova","year":"2019","journal-title":"Inf. Process. Manag."},{"key":"ref_6","unstructured":"Lin, C.Y. (2004). Proceedings of the Text Summarization Branches Out, Association for Computational Linguistics."},{"key":"ref_7","unstructured":"Ganesan, K. (2018). ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 6\u201312). Bleu: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_9","unstructured":"Tianyi, Z., Varsha, K., Felix, W., Kilian Q., W., and Yoav, A. (2020, January 26\u201330). BERTScore: Evaluating Text Generation with BERT. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia."},{"key":"ref_10","unstructured":"Kornilova, A., and Eidelman, V. (2019, January 4). BillSum: A Corpus for Automatic Summarization of US Legislation. Proceedings of the 2nd Workshop on New Frontiers in Summarization, Hong Kong, China."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Azzopardi, L., Stein, B., Fuhr, N., Mayr, P., Hauff, C., and Hiemstra, D. (2019, January 14\u201318). A Comparative Study of Summarization Algorithms Applied to Legal Case Judgments. Proceedings of the Advances in Information Retrieval, Cologne, Germany. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-030-15719-7"},{"key":"ref_12","unstructured":"Shen, Z., Lo, K., Yu, L., Dahlberg, N., Schlanger, M., and Downey, D. (2022). Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularities. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Xu, H., \u0160avelka, J., and Ashley, K.D. (2020, January 9\u201311). Using Argument Mining for Legal Text Summarization. Proceedings of the JURIX, Brno, Czech Republic.","DOI":"10.3233\/FAIA200862"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhong, L., Zhong, Z., Zhao, Z., Wang, S., Ashley, K.D., and Grabmair, M. (2019, January 17\u201321). Automatic Summarization of Legal Decisions using Iterative Masking of Predictive Sentences. Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law (ICAIL\u201919), Montreal, QC, Canada.","DOI":"10.1145\/3322640.3326728"},{"key":"ref_15","unstructured":"de Vargas Feij\u00f3, D., and Moreira, V.P. (2018). Lecture Notes in Computer Science, Springer International Publishing."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kryscinski, W., Keskar, N.S., McCann, B., Xiong, C., and Socher, R. (2019, January 3\u20137). Neural Text Summarization: A Critical Evaluation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1051"},{"key":"ref_17","unstructured":"Grusky, M., Naaman, M., and Artzi, Y. (2020). Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies. arXiv."},{"key":"ref_18","first-page":"2141","article-title":"Effective deep learning approaches for summarization of legal texts","volume":"34","author":"Anand","year":"2022","journal-title":"J. King Saud Univ.-Comput. Inf. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xu, H., Savelka, J., and Ashley, K.D. (2021, January 21\u201325). Toward summarizing case decisions via extracting argument issues, reasons, and conclusions. Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law, Sao Paulo, Brazil.","DOI":"10.1145\/3462757.3466098"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Papantoniou, K., and Tzitzikas, Y. (2020, January 2\u20134). NLP for the Greek Language: A Brief Survey. Proceedings of the 11th Hellenic Conference on Artificial Intelligence, ACM, Athens, Greece.","DOI":"10.1145\/3411408.3411410"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"113679","DOI":"10.1016\/j.eswa.2020.113679","article-title":"Automatic text summarization: A comprehensive survey","volume":"165","author":"Salama","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_22","unstructured":"Farzindar, A., and Lapalme, G. (2004). JURIX 2004, the Seventeenth Annual Conference, IOS Press."},{"key":"ref_23","unstructured":"Polsley, S., Jhunjhunwala, P., and Huang, R. (2016, January 11\u201316). CaseSummarizer: A System for Automated Summarization of Legal Texts. Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, Osaka, Japan."},{"key":"ref_24","unstructured":"Motomura, Y., Butler, A., and Bekki, D. (2013, January 27\u201328). Summarization of Legal Texts with High Cohesion and Automatic Compression Rate. Proceedings of the New Frontiers in Artificial Intelligence, Kanagawa, Japan. Lecture Notes in Computer Science."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Manor, L., and Li, J.J. (2019). Plain English Summarization of Contracts. arXiv.","DOI":"10.18653\/v1\/W19-2201"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Feijo, D., and Moreira, V. (2019, January 2\u20134). Summarizing Legal Rulings: Comparative Experiments. Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), Varna, Bulgaria.","DOI":"10.26615\/978-954-452-056-4_036"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1162\/tacl_a_00373","article-title":"SummEval: Re-evaluating Summarization Evaluation","volume":"9","author":"Fabbri","year":"2021","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Maynez, J., Narayan, S., Bohnet, B., and McDonald, R. (2020). On Faithfulness and Factuality in Abstractive Summarization. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.173"},{"key":"ref_29","unstructured":"Karampatzos, A., and Malos, G. (2015). Ius Comparatum-Global Studies in Comparative Law, Springer International Publishing."},{"key":"ref_30","unstructured":"Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., and Smith, N. (2020). Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping. arXiv."},{"key":"ref_31","unstructured":"Luijtgaarden, N.V.D. (2019). Automatic Summarization of Legal Text. [Master\u2019s Thesis, Utrecht University]. Available online: https:\/\/studenttheses.uu.nl\/handle\/20.500.12932\/34261."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1613\/jair.1523","article-title":"LexRank: Graph-based Lexical Centrality as Salience in Text Summarization","volume":"22","author":"Erkan","year":"2004","journal-title":"J. Artif. Intell. Res."},{"key":"ref_33","unstructured":"Page, L., Brin, S., Motwani, R., and Winograd, T. (1999). The PageRank Citation Ranking: Bringing Order to the Web, Stanford InfoLab. Technical Report 1999-66."},{"key":"ref_34","unstructured":"Mihalcea, R., and Tarau, P. (2004, January 25\u201326). TextRank: Bringing Order into Text. Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, Barcelona, Spain."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.ipm.2008.06.004","article-title":"Biased LexRank: Passage retrieval using random walks with question-based priors","volume":"45","author":"Otterbacher","year":"2009","journal-title":"Inf. Process. Manag."},{"key":"ref_36","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Liu, Y., and Lapata, M. (2019, January 3\u20137). Text Summarization with Pretrained Encoders. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1387"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Koutsikakis, J., Chalkidis, I., Malakasiotis, P., and Androutsopoulos, I. (2020, January 2\u20134). GREEK-BERT: The Greeks Visiting Sesame Street. Proceedings of the 11th Hellenic Conference on Artificial Intelligence (SETN 2020), Athens, Greece.","DOI":"10.1145\/3411408.3411440"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Allahyari, M., Pouriyeh, S., Assefi, M., Safaei, S., Trippe, E.D., Gutierrez, J.B., and Kochut, K. (2017). Text Summarization Techniques: A Brief Survey. Int. J. Adv. Comput. Sci. Appl., 8.","DOI":"10.14569\/IJACSA.2017.081052"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10462-016-9475-9","article-title":"Recent automatic text summarization techniques: A survey","volume":"47","author":"Gambhir","year":"2017","journal-title":"Artif. Intell. Rev."},{"key":"ref_41","unstructured":"Krippendorff, K. (2023, April 16). Computing Krippendorff\u2019s Alpha-Reliability. Available online: https:\/\/repository.upenn.edu\/asc_papers\/43."},{"key":"ref_42","unstructured":"Beltagy, I., Peters, M.E., and Cohan, A. (2020). Longformer: The Long-Document Transformer. arXiv."},{"key":"ref_43","unstructured":"Kitaev, N., Kaiser, L., and Levskaya, A. (2020). Reformer: The Efficient Transformer. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Liu, W., Wu, H., Mu, W., Li, Z., Chen, T., and Nie, D. (2021). CO2Sum: Contrastive Learning for Factual-Consistent Abstractive Summarization. arXiv.","DOI":"10.18653\/v1\/2021.findings-emnlp.106"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhu, C., Xu, R., Zeng, M., and Huang, X. (2020). A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining. arXiv.","DOI":"10.18653\/v1\/2020.findings-emnlp.19"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/4\/250\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:20:37Z","timestamp":1760124037000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/4\/250"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,21]]},"references-count":45,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["info14040250"],"URL":"https:\/\/doi.org\/10.3390\/info14040250","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,21]]}}}