Identificação de ameaças em fóruns da Dark Web e Surface Web: um estudo sobre a evolução temporal das discussões e generalização de modelos

Abstract

In light of the increasing structuring of cybercrime and the rise of coordinated threats in anonymous environments, Cyber Threat Intelligence (CTI) becomes essential for proactive defense. This dissertation proposes two case studies aimed at evaluating practical applications of CTI through the integration of data mining techniques in Surface Web and Dark Web forums. The first study investigates the temporal evolution of discussions between 2015 and 2024 using topic modeling with LDA, identifying seasonal patterns and a thematic transition from technical debates to criminal practices, such as the commercialization of personal data in the Portuguese language. The second study evaluates the effectiveness of transfer learning to overcome the scarcity of labeled data in the security domain. To this end, a model based on the LightGBM algorithm with TF-IDF representation was employed, previously developed by a member of the same research project, and applied across distinct domains and multilingual environments, Portuguese and English. The results demonstrate that the model exhibits generalization capability by isolating risk-related vocabularies in new sources, such as Dark Web marketplaces and generic discussion forums, although it shows high sensitivity to technical terms. This study contributes to the development of CTI models with high efficiency and adaptability across heterogeneous data sources, supporting the anticipation of incidents even in the presence of limited labeled data.