openai gym env.P, AttributeError 'TimeLimit' object has no attribute 'P'

Question

I'm currently reading Hands-On Reinforcement Learning with Python by Sudharsan Ravichandiran and on one of the first examples I run into this AttributeError:

AttributeError 'TimeLimit' object has no attribute 'P'

raised by the following line:

for next_sr in env.P[state][action]: 

I can't find any documentation regarding env.P, but I found a similar example written in python2 here: https://gym.openai.com/evaluations/eval_48sirBRSRAapMjotYzjb6w/

I suppose env.P is part of an outdated library (even if the the book was published on June 2018 and the incriminated code is in python3), so how can i replace it?

Answer 1

To get the maximum number of steps for an environment in newer versions of gym, you should use env.spec.max_episode_steps instead.

max_steps = args.max_timesteps or env.spec.max_episode_steps

Answer 2

If you are using a recent version of OpenAI Gym, the solution proposed in this github issue link worked for me.

As explained in the github issue, monitoring in the latest version of gym been replaced by wrappers, therefore monitoring will not work with the latest gym. To reimplement monitoring in a recent version of Gym, chang the code that resembles :

    env.monitor.start('cartpole-hill/', force=True)

to

    env = gym.wrappers.Monitor(env,directory='cartpole-hill/',force=True,write_upon_reset=True)

Answer 3

Try this,

for next_sr in env.env.P[state][action]:

Note the extra 'env' at start

For general use, try

>>> dir(class_name)

this will give list of member function.

Answer 4

Try to unwrap the env first by adding this

env = env.unwrapped