CODEWITHSUNDEEP
pythonpytorchgraphvizamazon-sagemakeramazon-sagemaker-studio
Edison
Edison

Reputation: 11987

Graphviz running on SageMaker notebook instance but not SageMaker Studio

I'm running a python script with PyTorch/Graphviz. It executes in a SageMaker notebook instance, but not in SageMaker Studio.

It appears the notebook instance with kernel conda_pytorch_p39 already contains an installation of Graphviz so the script just works as is and I get my Graphviz png.

When using SageMaker Studio with kernel PyTorch 1.8 Python 3.6 GPU, it seems to be a lot more complicated. Only installing torchviz didn't work so I tried installing graphviz via conda install -c fastchan python-graphviz.

I know that on Windows, Linux, macOS we need the path of graphviz executable but I didn't need to provide that in my SageMaker notebook instance.

SageMaker Studio Notebook (SageMaker kernel: PyTorch 1.8 Python 3.6 GPU)

The notebook I am debugging.

#Torchviz install
%pip install torchviz
[out]:
Installing collected packages: torchviz
Successfully installed torchviz-0.0.2
location: /opt/conda/lib/python3.6/site-packages/torchviz

# Tried installing python-graphviz
%conda install -c fastchan python-graphviz
[out]:
environment location: /opt/conda
  added / updated specs:
    - python-graphviz
Downloading and Extracting Packages
graphviz-2.42.3 
python-graphviz-0.16

from whych import whych
whych("graphviz")
[out]:
Python executable: /opt/conda/bin/python
Module "graphviz" found at location: /opt/conda/lib/python3.6/site-packages/graphviz

#However:
%cd /opt/conda/lib/python3.6/site-packages/graphviz
[out]:
No such file or directory
#It cannot find python3.6 even though the kernel is PyTorch 1.8 Python 3.6 GPU

# For testing only
import graphviz as gv
[out]:
AttributeError: module 'graphviz.backend' has no attribute 'ENCODING'

from torchviz import make_dot
make_dot(loss_tensor)

#Attempted fix:
import os
from torchviz import make_dot
os.environ['PATH'] += os.pathsep + '/opt/conda/lib/python3.6/site-packages/graphviz'
make_dot(loss_tensor)

[out]:
AttributeError: module 'graphviz.backend' has no attribute 'ENCODING'

#PATH
'/opt/amazon/openmpi/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tmp/miniconda3/condabin:/tmp/anaconda3/condabin:/tmp/miniconda2/condabin:/tmp/anaconda2/condabin'

Upvotes: 0

Views: 681

Answers (1)

Edison
Edison

Reputation: 11987

The SageMaker Studio image/kernel PyTorch 1.8 Python 3.6 GPU is not compatible with graphviz 0.20. I had to downgrade to 0.19.1 using pip install --force-reinstall graphviz==0.18. In the end I changed the SageMaker Studio kernel to Python 3.8 to upgrade to graphviz 0.20.1.

You can install graphviz 0.18.0 -> 0.19.1 with py3.6 and graphviz 0.20.0 -> 0.20.1 with py3.9.

In both SageMaker notebooks you do not need to add the path for graphviz like you do on Windows and macOS.

Upvotes: 0

Related Questions