Exploração de sistemas de gerenciamento de banco de dados vetoriais para dados financeiros

Abstract

This work aimed to demonstrate the feasibility of performing clustering and similarity search of financial data directly within a relational Database Management System (DBMS), expanding upon the working method of a previous study. The working method included the automated collection of NYSE (New York Stock Exchange) stock data, followed by preprocessing to handle missing values, remove duplicates, and normalize attributes. For the analysis, the data were converted into high-dimensional vectors and stored in a PostgreSQL DBMS, using the pgvector extension for native vector manipulation and similarity search tests. In addition, the K-Means algorithm was implemented as a stored procedure within the DBMS itself. The research results demonstrate that it is possible to efficiently perform clustering and similarity searches directly in the DBMS, eliminating the need to export large volumes of data to external environments and leveraging the optimized performance of the pgvector extension. It is concluded that the approach adopted in this thesis provides a robust and scalable solution for the analysis of large financial datasets.