Comparação entre algoritmos de aprendizado por reforço profundo e controlador PID aplicados ao seguimento de linha

Abstract

This work presents an experimental comparative evaluation of six deep reinforcement learning algorithms Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), Soft Actor-Critic (SAC), Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Model-Based Reinforcement Learning (MBRL) and a Proportional-Integral-Derivative (PID) controller, applied to the line-following task using computer vision on a differential mobile robot. The experiments were conducted in a simulated environment, considering three scenarios of increasing geometric complexity and two operating speeds, corresponding to 50% and 75% of the robot's maximum stable velocity. The evaluation considered metrics of precision, stability, control effort, completion rate, and robustness. The results indicate that DDPG achieved the best balance between accuracy and completion rate across all evaluated scenarios, maintaining consistent performance even as speed increased. The PID controller demonstrated competitive results under stable conditions, with lower control effort, but showed sensitivity to speed increases. The remaining reinforcement learning algorithms presented significantly higher tracking errors. Overall, the results reinforce the potential of deep reinforcement learning for continuous control tasks in mobile robotics, highlighting that the most suitable method depends on the specific requirements of the application.