Entre rimas e algoritmos: uma investigação sobre tradução automática poética

Abstract

Poetry translation is a complex task that extends beyond semantic transfer, requiring the preservation of rhythm, rhyme, imagery, stylistic elements, and cultural nuances. Despite advances in neural machine translation, these systems still show limitations in literary and poetic contexts. This work investigates the capabilities of automatic translation systems across six language pairs (Portuguese–English, English–Portuguese, Portuguese–French, French–Portuguese, English–French, and French–English), comparing specialized machine translation (MT) models, such as Google Translate, MarianMT, mBART, and OpenNMT (RNN), including fine-tuned versions trained on poetic and music lyrics corpora, with large language models (LLMs) such as ChatGPT and Maritaca AI. A three-phase evaluation pipeline was implemented, combining (i) automatic metrics (BLEU, METEOR, BERTScore, BARTScore) to assess lexical and semantic similarity, (ii) topic modeling with BERTopic to evaluate thematic preservation, and (iii) expert human evaluation focusing on poetic structure, style, fluency, and meaning. Results indicate that LLMs, and also Google Translate, consistently outperform traditional MT models, while OpenNMT with Recurrent Neural Network (RNN) performed worst. Prompt design influenced LLM performance: ChatGPT favored general prompts for higher semantic fidelity, whereas Maritaca AI better integrated explicit poetic instructions in portuguese. Human evaluation confirmed superior scores for LLMs in fluency and meaning preservation, though all models struggled with poetic Structure and stylistic. Topic modeling showed that top-performing models better preserved thematic consistency, often aligning more closely with human reference translations than with original poems. Fine-tuning effects were architecture-dependent: mBART benefited from poems and song lyrics, while MarianMT and OpenNMT (RNN) showed limited gains.