Exploring Social Dynamics in a Reinforcement Learning-based Metaheuristic: A study using Improvement Frequency and Population Turnover

Resumo

Swarm Intelligence (SI) metaheuristics have been extensively employed in optimisation, yet their social dynamics analysis has not been explored sufficiently. This paper addresses this gap by investigating inner dynamics in an RL-based metaheuristic using two recently proposed metrics, Improvement Frequency and Population Turnover. The goal is to reveal the underlying dynamics that rule the agents of the metaheuristic compared with other well-known metaheuristics. Our findings indicate that RL can effectively combine or replicate the social dynamics available across metaheuristics for multiple scenarios, making possible the creation of adaptive metaheuristics shaped for specific problems. We observed that the search behaviours exhibited by the RL-metaheuristics were closely related to the specific benchmark problems used in the experiments. Thus, our findings support the idea of using RL to create effective metaheuristics for problems that are still unknown – no clear evidence of which algorithm would be adequate.