CODEWITHSUNDEEP
google-cloud-vertex-ai
tibfab
tibfab

Reputation: 21

GCP Vertex AI Managed Notebook cannot use custom container

In GCP Vertex AI, I created a Managed Notebook by specifying one of our custom containers which work perfectly with User-Managed Notebook kernels. The Managed Notebook starts, and Jupyter Lab seems to work without any signs of error.

Unfortunately, if I look at the available kernels in Jupyter Lab, only the default kernels are listed but not my custom kernel.

An activity log entry on the right shows a spinning wheel "Loading kernel from [custom container]" which never disappears.
Taking a look at the terminal,

docker image ls

does not show the custom container either; obviously, it was not even pulled to the Managed Notebook.

If I perform

docker pull [custom container]

in the terminal, to test connectivity to the Artifact Registry then it pulls the container correctly as expected. However, the custom kernel is still not visible in Jupyter Lab (even after a notebook restart).

Upvotes: 2

Views: 1314

Answers (1)

Martynas Jurkus
Martynas Jurkus

Reputation: 9301

I faced the same issue today, and after some experimentation, I found that I haven't fulfilled all the requirements in Vertex AI docs.
One in particular - The Docker container image must support sleep infinity.

And my Dockerfile contained

EXPOSE 8080

ENTRYPOINT [ "jupyter", "lab", "--allow-root", "--ip", "0.0.0.0", "--config", "/opt/jupyter/.jupyter/jupyter_notebook_config.py" ]

To enable sleep infinity I replaced ENTRYPOINT with

ENTRYPOINT ["/bin/sh", "-c", "sleep infinity"]

then the kernelspec was imported, but the notebook wasn't able to connect.

So I removed the ENTRYPOINT entirely and then the kernelspec was imported and the notebook was able to connect successfully.

Hope that helps.

Upvotes: 1

Related Questions