Efficient Reward Shaping for Multiagent Systems

In this article, we address the reward-shaping problem of large-scale multiagent systems (MASs) using inverse reinforcement learning (IRL). The learning MAS does not have prior knowledge of the cost function of the target MAS and aims to reconstruct it based on the target's demonstrations. We propose a scalable model-free IRL algorithm for a large-scale MAS, where dynamic mode decomposition (DMD) extracts dynamic modes and builds a projection matrix. This significantly reduces the data required while retaining the system's essential dynamic information. The proofs of the algorithm's convergence, stability, and nonuniqueness of the state reward weight are presented. The efficacy of our method is validated with a large-scale consensus network, by comparing the required data sizes and computational time for reward shaping with and without DMD.