Análise Comparativa de LLMs para Detecção de Racismo, Sexismo e Homofobia em Redes Sociais

Abstract

This study investigated the effectiveness of large language models (LLMs), including GPT-3.5-Turbo, GPT-4.0, DeepSeek-V3, and Gemini-2.0-Flash, in detecting hate speech on social media, focusing on three main categories: homophobia, sexism, and racism. The primary objective was to compare the performance of these models in real-world scenarios, analyzing both raw and pre-processed comments to identify the optimal balance between accuracy, cost, and computational efficiency. The methodology involved collecting data from real-world datasets, totaling over 2,000 manually labeled comments. Pre-processing techniques were then applied to assess their impact on detecting linguistic nuances, and standardized prompts were implemented for classification. The results indicated that DeepSeek-V3 provided the best balance between performance and cost, achieving an average F1-score of 93.37% and excellent performance on homophobia (99.63%), with a cost of US$ 0.66, significantly lower than GPT-4.0 (US$ 26.99). Although GPT-4.0 achieved the highest overall accuracy (94.19%), its high cost makes large-scale use impractical. Gemini-2.0-Flash, while the most economical (US$ 0.12), showed inconsistent performance, particularly on pre-processed racism comments (46.15% F1-score). It was observed that traditional pre-processing generally reduced the F1-score by 6.93%, except for the racism category in GPT models, where detection improved.