Cooperative Multi-Agent Reinforcement Learning with Constraint-Reduced DCOP

Cooperative multi-agent reinforcement learning (MARL) is an important topic in the field of artificial intelligence, in which distributed constraint optimization (DCOP) algorithms have been widely used to coordinate the actions of multiple agents. However, dense communication among agents affects the practicability of DCOP algorithms. In this paper, we propose a novel DCOP algorithm dealing with the previous DCOP algorithms' communication problem by reducing constraints. The contributions of this paper are primarily threefold:① It is proved that removing constraints can effectively reduce the communication burden of DCOP algorithms. ② An criterion is provided to identify insignificant constraints whose elimination doesn't have a great impact on the performance of the whole system. ③ A constraint-reduced DCOP algorithm is proposed by adopting a variant of spectral clustering algorithm to detect and eliminate the insignificant constraints. Our algorithm reduces the communication burdern of the benchmark DCOP algorithm while keeping its overall performance unaffected. The performance of constraint-reduced DCOP algorithm is evaluated on four configurations of cooperative sensor networks. The effectiveness of communication reduction is also verified by comparisons between the constraint-reduced DCOP and the benchmark DCOP.