Heterogeneous Multi-Agent Reinforcement Learning based on Modularized Policy Network

Abstract

Rapid advancements in multi-agent systems enable multiple robots to operate cooperatively. However, conventional methods based on homogeneous multi-agent reinforcement learning (MARL) struggle to generate distinct behavioral responses for each agent and often result in divergent model learning. Therefore, we exploit the heterogeneous MARL algorithm with a novel modularized policy network that can consider the agents’ heterogeneity while learning cooperative tasks. Furthermore, each module network can be separately trained for its role, improving adaptation in new environments. We conduct experiments to demonstrate the effect of modularized network-based policy on enhancing the heterogeneity of teams and adapting well to unseen environments or scenarios.