CODEWITHSUNDEEP
pythonopenai-gymstable-baselines
nize
nize

Reputation: 1032

What is missing in OpenAI Gym registration?

I get ValueError: xxx not found in gym registry, you maybe meant when trying to register a custom environment in stable baselines 3. I tried the following commands:

apt-get install swig cmake ffmpeg freeglut3-dev xvfb
git clone --recursive https://github.com/DLR-RM/rl-baselines3-zoo
cd rl-baselines3-zoo
pip3 install -r requirements.txt
cd ..
git clone https://github.com/MatePocs/gym-basic.git
cd gym-basic
pip3 install -e .
cd ..
cd rl-baselines3-zoo
python3 train.py --algo td3 --env basic-v0 --eval-freq 1000 --save-freq 5000

The result is:

Traceback (most recent call last):
  File "train.py", line 107, in <module>
    raise ValueError(f"{env_id} not found in gym registry, you maybe meant {closest_match}?")
ValueError: basic-v0 not found in gym registry, you maybe meant CubeCrash-v0?

Do you see which is the mistake?

Upvotes: 1

Views: 1401

Answers (2)

furas
furas

Reputation: 142651

This problem can be resolved with --gym-packages gym_basic

python3 train.py --algo td3 --env basic-v0 --eval-freq 1000 --save-freq 5000 --gym-packages gym_basic

But then it shows other problem with hyperparameters.
For standard environments it has predefined tunned hyperparameters in folder hyperparameters. For --algo td3 in hyperparameters/td3.yml.
But there is no parameters for basic-v0

If I duplicate other parameters and use name basic-v0

basic-v0:
  n_timesteps: 300000
  policy: 'MlpPolicy'
  noise_type: 'ornstein-uhlenbeck'
  noise_std: 0.5

then (theoretically) it resolves this problem

... but it gives next problem.

train want to get env.action_space.shape[0] and basic-v0 doesn't have [0] in shape

So all this many need more changes.

Upvotes: 1

Andrew Tapia
Andrew Tapia

Reputation: 1446

Gym doesn't know about your gym-basic environment—you need to tell gym about it by importing gym_basic.

For the train.py script you are running from RL Baselines3 Zoo, it looks like the recommended way is to import your custom environment in utils/import_envs.py. You should append something like the following to that file.

try:
    import gym_basic
except ImportError:
    gym_basic = None

You then need to set the hyperparameters for the environment for the agent you are using—in this case, td3—in the relevant file under the hyperparameters directory. You can look at the existing environment hyperparameters in the hyperparameters/td3.yml file to get a sense of what hyperparameters must be specified. You can then set your hyperparameters for the basic-v0 environment like this.

basic-v0:
  # Hyperparameters go here...

However, you should note that, according to the Stable Baselines 3 documentation documentation, td3 does not support discrete action spaces, so training will likely fail even after you specify your hyperparameters.

You might want to try a different algorithm—the steps should be the same as those described above. Just specify a different algorithm when running train.py, and be sure to specify the hyperparameters in the relevant file under the hyperparameters directory.

Upvotes: 2

Related Questions