MARL Papers with Code

This is a collection of Multi-Agent Reinforcement Learning (MARL) papers with code. I have selected some relatively important papers with open source code and categorized them by time and method.

For MARL papers and MARL resources, please refer to Multi Agent Reinforcement Learning papers and MARL Resources Collection.

I will continually update this repository and I welcome suggestions. (missing important papers, missing categories, invalid links, etc.) This is only a first draft so far and I'll add more resources in the next few months.

This repository is not for commercial purposes.

My email: [email protected]

Overview

• Classic Papers
• Other Papers
• TODO

Classic Papers

Algorithms

Category	Paper	Code	Accepted at	Year
Independent Learning	IQL：Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents	https://github.com/oxwhirl/pymarl	ICML	1993
Value Decomposition	VDN：Value-Decomposition Networks For Cooperative Multi-Agent Learning	https://github.com/oxwhirl/pymarl	AAMAS	2017
Value Decomposition	QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning	https://github.com/oxwhirl/pymarl	ICML	2018
Value Decomposition	QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning	https://github.com/oxwhirl/pymarl	ICML	2019
Policy Gradient	COMA：Counterfactual Multi-Agent Policy Gradients	https://github.com/oxwhirl/pymarl	AAAI	2018
Policy Gradient	MADDPG：Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments	https://github.com/openai/maddpg	NIPS	2017
Communication	BiCNet：Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games	https://github.com/Coac/CommNet-BiCnet		2017
Communication	CommNet：Learning Multiagent Communication with Backpropagation	https://github.com/facebookarchive/CommNet	NIPS	2016
Communication	IC3Net：Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks	https://github.com/IC3Net/IC3Net		2018
Communication	RIAL/RIDL：Learning to Communicate with Deep Multi-Agent Reinforcement Learning	https://github.com/iassael/learning-to-communicate	NIPS	2016
Exploration	MAVEN：Multi-Agent Variational Exploration	https://github.com/starry-sky6688/MARL-Algorithms	NIPS	2019

Environments

Environment	Paper	Code	Accepted at	Year
StarCraft	The StarCraft Multi-Agent Challenge	https://github.com/oxwhirl/smac	NIPS	2019
StarCraft	SMACv2: A New Benchmark for Cooperative Multi-Agent Reinforcement Learning	https://github.com/oxwhirl/smacv2		2022
StarCraft	Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks	https://github.com/uoe-agents/epymarl	NIPS	2021
Football	Google Research Football: A Novel Reinforcement Learning Environment	https://github.com/google-research/football	AAAI	2020
PettingZoo	PettingZoo: Gym for Multi-Agent Reinforcement Learning	https://github.com/Farama-Foundation/PettingZoo	NIPS	2021
Melting Pot	Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot	https://github.com/deepmind/meltingpot	ICML	2021
MuJoCo	MuJoCo: A physics engine for model-based control	https://github.com/deepmind/mujoco	IROS	2012
MALib	MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning	https://github.com/sjtu-marl/malib		2021
MAgent	MAgent: A many-agent reinforcement learning platform for artificial collective intelligence	https://github.com/Farama-Foundation/MAgent	AAAI	2018
Neural MMO	Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents	https://github.com/openai/neural-mmo		2019
MPE	Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments	https://github.com/openai/multiagent-particle-envs	NIPS	2017
Pommerman	Pommerman: A multi-agent playground	https://github.com/MultiAgentLearning/playground		2018
HFO	Half Field Offense: An Environment for Multiagent Learning and Ad Hoc Teamwork	https://github.com/LARG/HFO	AAMAS Workshop	2016

Other Papers

Category	Paper	Code	Accepted at	Year
Graph Neural Network	Multi-Agent Game Abstraction via Graph Attention Neural Network	https://github.com/starry-sky6688/MARL-Algorithms	AAAI	2020
Curriculum Learning	From Few to More: Large-Scale Dynamic Multiagent Curriculum Learning	https://github.com/starry-sky6688/MARL-Algorithms	AAAI	2020
Curriculum Learning	EPC：Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning	https://github.com/qian18long/epciclr2020	ICLR	2020
Curriculum Learning/Emergent	Emergent Tool Use From Multi-Agent Autocurricula	https://github.com/openai/multi-agent-emergence-environments	ICLR	2020
Curriculum Learning	Cooperative Multi-agent Control using deep reinforcement learning	https://github.com/sisl/MADRL	AAMAS	2017
Role	ROMA: Multi-Agent Reinforcement Learning with Emergent Roles	https://github.com/TonghanWang/ROMA	ICML	2020
Role	RODE: Learning Roles to Decompose Multi-Agent Tasks	https://github.com/TonghanWang/RODE	ICLR	2021
Role	Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing	https://github.com/uoe-agents/seps	ICML	2021
Opponent Modeling	Opponent Modeling in Deep Reinforcement Learning	https://github.com/hhexiy/opponent	ICML	2016
Selfish Agent	M3RL: Mind-aware Multi-agent Management Reinforcement Learning	https://github.com/facebookresearch/M3RL	ICLR	2019
Communication	Emergence of grounded compositional language in multi-agent populations	https://github.com/bkgoksel/emergent-language	AAAI	2018
Communication	Fully decentralized multi-agent reinforcement learning with networked agents	https://github.com/cts198859/deeprl_network	ICML	2018
Policy Gradient	DOP: Off-Policy Multi-Agent Decomposed Policy Gradients	https://github.com/TonghanWang/DOP	ICLR	2021
Policy Gradient	MAAC：Actor-Attention-Critic for Multi-Agent Reinforcement Learning	https://github.com/shariqiqbal2810/MAAC	ICML	2019
Environment	Emergent Complexity via Multi-Agent Competition	https://github.com/openai/multiagent-competition	ICLR	2018
Exploration	EITI/EDTI：Influence-Based Multi-Agent Exploration	https://github.com/TonghanWang/EITI-EDTI	ICLR	2020
Exploration	LIIR: Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning	https://github.com/yalidu/liir	NIPS	2019
From Single-Agent to Multi-Agent	MAPPO：The Surprising Effectiveness of MAPPO in Cooperative, Multi-Agent Games	https://github.com/marlbenchmark/on-policy		2021
Diversity	Q-DPP：Multi-Agent Determinantal Q-Learning	https://github.com/QDPP-GitHub/QDPP	ICML	2020
Ad Hoc Teamwork	CollaQ：Multi-Agent Collaboration via Reward Attribution Decomposition	https://github.com/facebookresearch/CollaQ		2020
Value Decomposition	NDQ: Learning Nearly Decomposable Value Functions Via Communication Minimization	https://github.com/TonghanWang/NDQ	ICLR	2020
Value Decomposition	QPLEX: Duplex Dueling Multi-Agent Q-Learning	https://github.com/wjh720/QPLEX	ICLR	2021
Self-Play	TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning	https://github.com/tencent-ailab/TLeague		2020
Transformer	UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers	https://github.com/hhhusiyi-monash/UPDeT	ICLR	2021
Sparse Reward	Individual Reward Assisted Multi-Agent Reinforcement Learning	https://github.com/MDrW/ICML2022-IRAT	ICML	2022
Ad Hoc	Open Ad Hoc Teamwork using Graph-based Policy Learning	https://github.com/uoe-agents/GPL	ICLM	2021
Generalization	UNMAS: Multiagent Reinforcement Learningfor Unshaped Cooperative Scenarios	https://github.com/James0618/unmas	TNNLS	2021
Other	SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learning	https://github.com/deligentfool/SIDE	AAMAS	2022
Other	Context-Aware Sparse Deep Coordination Graphs	https://github.com/TonghanWang/CASEC-MACO-benchmark	ICLR	2022

TODO

• More papers will be added to Other Papers.

Citation

If you find this repository useful, please cite our repo:

@misc{chen2021multi,
  author={Chen, Hao},
  title={Multi-Agent Reinforcement Learning Papers with Code},
  year={2021}
  publisher = {GitHub},
  journal = {GitHub Repository},
  howpublished = {\url{https://github.com/TimeBreaker/MARL-papers-with-code}}
}