A Multidimensional Analysis of Swarm Dynamics in a Reinforcement Learning-based Metaheuristic

Resumo

Traditional assessments of metaheuristics typically focus on performance metrics when solving benchmark problems, with limited attention to the underlying mechanisms that drive collective intelligence. This performance-centric approach fails to indicate why certain metaheuristics are successful in specific scenarios and provides minimal guidance for improving their design. To better understand these inner dynamics, we conducted a multidimensional analysis of the swarm dynamics in a Reinforcement Learning-based metaheuristic that can combine search behaviours from different Swarm Intelligence metaheuristics. Using Interaction Networks, Search Trajectory Networks, Improvement Frequency, and Population Turnover metrics simultaneously, we reveal how the RL approach dynamically switches between different swarm behaviours depending on the problem. Our results demonstrate that the RL-based metaheuristic matches the performance of the most effective specialised algorithm for each benchmark function by adaptively mimicking its social interaction patterns, search trajectories, and fitness development. We show that for the F1 function, the RL approach adopts behaviors similar to GWO, while for the F2 function, it transitions to GPSO-like patterns, aligning with the expected best-performing characteristics reported in the literature. This multidimensional approach provides insights into how adaptive metaheuristics combine different behaviours at optimisation stages, suggesting a pathway toward self-adaptive metaheuristics that automatically select optimal search strategies according to the simulation scenario.