Uma nova abordagem de aprendizagem de máquina combinando elicitação automática de casos, aprendizagem por reforço e mineração de padrões sequenciais para agentes jogadores de damas

Abstract

ake into account, in addition to the environment, the minimizing action of an opponent (such as in games), it is fundamental that the agent has the ability to progressively trace a proĄle of its adversary that aids it in the process of selecting appropriate actions. However, it would be unsuitable to construct an agent with a decision-making system based on only the elaboration of this proĄle, as this would prevent the agent from having its Şown identityŤ, which would leave it at the mercy of its opponent. Following this direction, this work proposes an automatic hybrid Checkers player, called ACE-RL-Checkers, equipped with a dynamic decision-making mechanism, which adapts to the proĄle of its opponent over the course of the game. In such a system, the action selection process (moves) is conducted through a composition of Multi-Layer Perceptron Neural Network and case library. In the case, Neural Network represents the ŞidentityŤ of the agent, i.e., it is an already trained static decision-making module and makes use of the Reinforcement Learning TD( ) techniques. On the other hand, the case library represents the dynamic decision-making module of the agent, which is generated by the Automatic Case Elicitation technique (a particular type of Case-Based Reasoning). This technique has a pseudo-random exploratory behavior, which makes the dynamic decision-making on the part of the agent to be directed, either by the game proĄle of the opponent or randomly. However, when devising such an architecture, it is necessary to avoid the following problem: due to the inherent characteristics of the Automatic Case Elicitation technique, in the game initial phases, in which the quantity of available cases in the library is extremely low due to low knowledge content concerning the proĄle of the adversary, the decisionmaking frequency for random decisions is extremely high, which would be detrimental to the performance of the agent. In order to attack this problem, this work also proposes to incorporate onto the ACE-RL-Checkers architecture a third module composed of a base of experience rules, extracted from games played by human experts, using a Sequential Pattern Mining technique. The objective behind using such a base is to reĄne and accelerate the adaptation of the agent to the proĄle of its opponent in the initial phases of their confrontations. Experimental results conducted in tournaments involving ACE-RL-Checkers and other agents correlated with this work, conĄrm the superiority of the dynamic architecture proposed herein.