Geração e detecção de textos artificiais em redes sociais por meio de modelos de linguagem de grande escala

Abstract

The increasing use of Large Language Models (LLMs) has brought about the need for discussions and analyses regarding their impact on digital platforms, such as social networks, due to the difficulty in distinguishing content generated by such technologies. This work explores the use of LLMs in simulating human interactions in the digital environment, specifically on platform X (formerly Twitter), and analyzes the effectiveness of detectors found in the literature in distinguishing texts generated by the selected models, as well as investigating how the use of predefined personas influences the structure of the texts and their detection based on linguistic metrics. The methodology of this work involves collecting real posts extracted from X based on selected topics. Personas were developed and adopted by the language models to generate artificial comments related to each post. From this generated data, various detectors from the literature were applied, resulting in analyses related to the effectiveness of both the generation and detection of the comments. In addition to these tests, linguistic metrics were applied to uncover structural similarities among synthetic texts, original texts from X, and how the absence of personas affects the outcome of the metrics and detections. To automate the workflow, a web tool and some scripts were developed to integrate the experimental processes. The results demonstrated that LLMs are able to satisfactorily replicate human language patterns present on social networks, using informality and linguistic variations, making it difficult to distinguish between real and AI-generated text. AI detectors proved to be relevant tools for such distinction. Furthermore, it was observed that the use of personas and variation in generation parameters subtly influence the structure of the texts, as well as the complexity of their detection. These results demonstrate the evolution of detection techniques through various approaches. This work contributes to the understanding of the capabilities of current language models and detection tools, as well as highlighting the continuous need for technological advances in the field of artificial content detection due to its ethical, social, and cultural impacts.