Sistema estruturado e híbrido de recomendação de problemas de programação no Codeforces: perfil multitag e recuperação semântica

Abstract

Online Judge platforms provide extensive catalogs of programming problems, but selecting activities suited to each competitor’s experience can be challenging, particularly because proficiency may vary across topics. This study aimed to develop and evaluate a hybrid Codeforces problem recommender system based on tag-specific proficiency profiles, structured filters, semantic retrieval through embedding-based vector search, and optional curation using a Large Language Model (LLM). The evaluation used public records from 85 competitors, whose solved-problem histories were chronologically divided into an observed past and an observed future. The observed past was used to identify the three most frequent tags, calculate the operational proficiency level for each tag, and generate recommendations. Four approaches were compared: structured retrieval without an LLM, structured retrieval with an LLM, hybrid retrieval without an LLM, and hybrid retrieval with an LLM. In total, 340 recommendation lists comprising 3,400 recommendations were evaluated. The hybrid approaches achieved a Hit Rate@10 of 44.71%, whereas the structured approaches achieved 9.41% and 10.59%. In the hybrid setting, the use of the LLM preserved the same Hit Rate@10, increased mean semantic diversity from 0.2121 to 0.2169, and expanded the overall variety from 207 to 221 distinct problems. All recommendations remained within the difficulty range defined for each tag. The results supported all three research hypotheses in the evaluated scenario: the hybrid procedure produced greater correspondence with the observed future, LLM-based curation increased the diversity and variety of the recommendation lists, and the structured filters preserved difficulty control.