CODEWITHSUNDEEP
azure-machine-learning-service
kohgushi
kohgushi

Reputation: 31

Does AzureML RL support PyTorch?

Does AzureML RL support PyTorch?

As RLlib itself supports PyTorch as a framework, I tried to run AzureML RL with PyTorch but it failed.

I referred to this page to know how to specify the framework.

I added "framework":"torch" to my AzureML RL experiment's config but it failed.

Here're snippet from the training script.

tune.run(
    run_or_experiment="PPO",
    config={
        "env":"CartPole-v0",
        "env_config":env_config,
        "num_gpus":0,
        "num_workers":1,
        "callbacks":callbacks,
        "framework": "torch",
    },
    stop=stop,
    checkpoint_freq=2,
    checkpoint_at_end=True,
    local_dir='./logs',

Upvotes: 1

Views: 88

Answers (1)

Dan Sommerfield
Dan Sommerfield

Reputation: 21

Ray's support for PyTorch exists, but is not nearly as extensive as its support for Tensorflow.

Whether or not PyTorch will work for your problem depends on the version of Ray/RLLib you're using, the algorithm you're running, and sometimes even the nature of the Environment (specifically the action and observation spaces).

I recommend starting by making sure you're using a recent version of Ray. You can select a version by specifying a Pip package in the configuration for your ReinforcementLearningEstimator (this will be in your notebook code, not in the training script). You can add code that looks something like this:

pip_packages=["ray[rllib]==0.8.7"]

Then in your ReinforcementLearningEstimator setup make sure you set pip_packages:

rl_estimator = ReinforcementLearningEstimator(    
    ...
    # Pip packages
    pip_packages=pip_packages,  
    ...

Upvotes: 2

Related Questions