CODEWITHSUNDEEP
reinforcement-learningopenai-gym
Martin Studer
Martin Studer

Reputation: 2321

Openai gym environment for multi-agent games

Is it possible to use openai's gym environments for multi-agent games? Specifically, I would like to model a card game with four players (agents). The player scoring a turn starts the next turn. How would I model the necessary coordination between the players (e.g. who's turn it is next)? Ultimately, I would like to use reinforcement learning on four agents that play against each other.

Upvotes: 34

Views: 20187

Answers (4)

Adrien Forbu
Adrien Forbu

Reputation: 431

What you are looking for is PettingZoo, it's a set of environment with multi agent setting and they have a specific class / synthax to handle multi agent environment.

It's an interesting library because you can also use it with ray / rllib to use already implemented algorithm like PPO / Q-learning. Like in this exemple.

Rllib also have an implementation for multiagents environments. But you will have to dig deeper in the documentation to understand it.

Upvotes: 6

Jon Deaton
Jon Deaton

Reputation: 4359

Yes, it is possible to use OpenAI gym environments for multi-agent games. Although in the OpenAI gym community there is no standardized interface for multi-agent environments, it is easy enough to build an OpenAI gym that supports this. For instance, in OpenAI's recent work on multi-agent particle environments they make a multi-agent environment that inherits from gym.Env which takes the following form:

class MultiAgentEnv(gym.Env):

    def step(self, action_n):
        obs_n    = list()
        reward_n = list()
        done_n   = list()
        info_n   = {'n': []}
        # ...
        return obs_n, reward_n, done_n, info_n

We can see that the step function takes a list of actions (one for each agent) and returns a list of observations, list of rewards, list of dones, while stepping the environment forwards. This interface is representative of Markov Game, in which all agents take actions at the same time and each observe their own subsequent observation, reward.

However, this kind of Markov Game interface may not be suitable for all multi-agent environments. In particular, turn-based games (such as card games) might be better cast as an alternating Markov Game, in which agents take turns (i.e. actions) one at a time. For this kind of environment, you may need to include which agent's turn it is in the representation of state, and your step function would then just take a single action, and return a single observation, reward and done.

Upvotes: 32

MathKid
MathKid

Reputation: 2112

There is a specific multi-agent environment for reinforcement learning here. It supports any number of agents written in any programming language. An example game is already implemented which happens to be a card game.

Upvotes: 0

vijay singh Rajpurohit
vijay singh Rajpurohit

Reputation: 189

There is a multi-agent deep deterministic policy gradient MADDPG approach has been implemented by OpenAI team.

This is the repo to get started. https://github.com/openai/multiagent-particle-envs

Upvotes: 2

Related Questions