Authors:
Beatriz Ribeiro Borges
;
Paulo Henrique Ribeiro Gabriel
and
Elaine Ribeiro de Faria
Affiliation:
Faculty of Computer Science, Federal University of Uberlândia, Uberlândia, Brazil
Keyword(s):
Poetry Translation, Machine Translation, Automatic Translation, Large Language Models, Literary Translation.
Abstract:
This study investigates the performance of specialized machine translation (MT) models and large language models (LLMs) in the automatic translation of poetry across six language pairs (FR-EN, FR-PT, EN-FR, EN-PT, PT-FR, PT-EN). Automatic evaluations using BLEU, METEOR, and BERTScore were complemented by human assessments focusing on poetic structure, stylistics, fluency, meaning, and overall impression. Results indicate that LLMs, particularly ChatGPT overall and Maritaca AI for translations into Portuguese, outperform specialized MT models in semantic fidelity and fluency, although Google Translate also performed very well, surpassing other MT models such as MarianMT, mBART, and OpenNMT (RNN). Despite these successes, all models struggle with poetic form, rhyme, and figurative nuances.