StableBaselines creating a model segmentation fault

Question

I am getting a segmentation fault when trying to create a stable_baselines3 PPO model on a CartPole-v1 OpenAI Gym environment.

So far what I've tried is running a short example code on Python 3.10 as well as Python 3.9. I'm running the python script in a Conda environment. What I did was install stable-baselines[extra] using pip. I also installed the openAI gym library using conda.

With Python 3.10 I was getting a Segmentation fault in the pthreading.py file with a call to wait function call. Using Python 3.9 I'm getting a different error in the constraints.py file in the _IntegerInterval.check method

This is the example code:

import sys
import gym
from stable_baselines3 import PPO, A2C
from stable_baselines3.common.env_util import make_vec_env

import faulthandler

faulthandler.enable()

def main(): 
  print("Going to create model")
  env = gym.make("CartPole-v1")

  model = PPO("MlpPolicy", env, verbose=1)
  print("Model created")
  model.learn(total_timesteps=25000)
  model.save("ppo_cartpole")

if __name__ == '__main__':
    main()
    sys.settrace(None)

This is the terminal output:

(py39) ilijas-mbp:Doom_DQN_GC ilijavuk$  cd /Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC ; /usr/bin/env /Users/ilijavuk/opt/anaconda3/envs/py39/bin/python /Users/ilijavuk/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 50143 -- /Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC/StableBaselinesTest.py 
Going to create model
Backend MacOSX is interactive backend. Turning interactive mode on.
Using cpu device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.
Fatal Python error: Segmentation fault

Thread 0x0000700007a7b000 (most recent call first):
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/threading.py", line 316 in wait
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/threading.py", line 581 in wait
  File "/Users/ilijavuk/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 261 in _on_run
  File "/Users/ilijavuk/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
(py39) ilijas-mbp:Doom_DQN_GC ilijavuk$ 

So it seems like the model creation is failing somewhere in the DummyVecEnv wrapping step

EDIT: I managed to run the test script with gdb. This is the current trace that I'm getting:

Starting program: /Users/ilijavuk/opt/anaconda3/envs/py39/bin/python StableBaselinesTest.py
[New Thread 0x1603 of process 2803]
[New Thread 0x2003 of process 2803]
warning: unhandled dyld version (17)
Going to create model
Using cpu device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.
Model created
[New Thread 0x1807 of process 2803]
[New Thread 0x2103 of process 2803]
[New Thread 0x2203 of process 2803]
[New Thread 0x2303 of process 2803]
[New Thread 0x2403 of process 2803]
[New Thread 0x2503 of process 2803]
[New Thread 0x2603 of process 2803]
[New Thread 0x2703 of process 2803]
[New Thread 0x2803 of process 2803]
[New Thread 0x2903 of process 2803]
[New Thread 0x2a03 of process 2803]
[New Thread 0x2b03 of process 2803]
[New Thread 0x3e03 of process 2803]
[New Thread 0x3f03 of process 2803]

Thread 3 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x1807 of process 2803]
0x0000000000000000 in ?? ()
(gdb) backtrace
#0  0x0000000000000000 in ?? ()
#1  0x0000000103a48538 in ?? ()
#2  0x0000000000000000 in ?? ()
(gdb)

After quitting gdb I also get this. HOWEVER, this seems to be tied to the model.learn call, since this error disappears when I comment model.learn and model.save out

(gdb) q
A debugging session is active

        Inferior 1 [process 2803] will be killed.

Quit anyway? (y or n) y
Fatal Python error: Segmentation fault

Thread 0x00007ff847bb84c0 (most recent call first):
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/linear.py", line 114 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/container.py", line 204 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/torch_layers.py", line 263 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/policies.py", line 627 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 166 in collect_rollouts
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 248 in learn
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/ppo/ppo.py", line 307 in learn
  File "/Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC/StableBaselinesTest.py", line 16 in main
  File "/Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC/StableBaselinesTest.py", line 20 in <module>

Answer 1

Try reverting to the earlier version of Python (such as 3.7.0)