CODEWITHSUNDEEP
pythonreinforcement-learningopenai-gymq-learning
Möbius
Möbius

Reputation: 27

When the action is to move right in CartPole, it moves to the left side. Why it is like that? How can this be resolved?

In my experiments, I'm using OpenAI's CartPole-v1 environment. I need to set a state and then perform an action on that state. When I perform a specific action, it does not behave as expected. For example, when the action "Go Right" is used, it moves to the left.

I have a state s=[ 0.048 0.151 -0.037 -0.265] and an action: Go Right which is 1 according to document.

When I applied the action 1 to the s, I am getting the new state [-0.016 0.206 -0.029 -0.326]

It is supposed to move the right side.

Here is the code I tried:

env.reset()
state=[ 0.048  0.151 -0.037 -0.265]
env.state = env.unwrapped.state = state
s, reward, done, _ = env.step(action)
print(state,s)

Upvotes: 1

Views: 96

Answers (1)

Kien
Kien

Reputation: 11

It's because of the cart's velocity. An action would affect the cart's velocity instead of its position. Velocity, in turns, would decide the cart's position in the next state. The velocity needs to reverse its sign (neg/pos) first before the cart could move in that corresponding direction (neg -> go left, pos -> go right).

For example, your action is to "Go left" but your velocity is still positive, the cart would still "Go right" until velocity turns negative.

Upvotes: 1

Related Questions