RLlib implementation of Foerster, Jakob N., et al. "Counterfactual Multi-Agent Policy Gradients." (2018).