Vertex AI custom prediction vs Google Kubernetes Engine

Question

I have been exploring using Vertex AI for my machine learning workflows. Because deploying different models to the same endpoint utilizing only one node is not possible in Vertex AI, I am considering a workaround. With this workaround, I will be unable to use many Vertex AI features, like model monitoring, feature attribution etc., and it simply becomes, I think, a managed alternative to running the prediction application on, say, a GKE cluster. So, besides the cost difference, I am exploring if running the custom prediction container on Vertex AI vs. GKE will involve any limitations, for example, only N1 machine types are available for prediction in Vertex AI

There is a similar question, but I it does not raise the specific questions I hope to have answered.

• I am not sure of the available disk space. In Vertex AI, one can specify the machine type, such as n1-standard-2 etc., but I am not sure what disk space will be available and if/how one can specify it? In the custom container code, I may copy multiple model artifacts, or data from outside sources to the local directory before processing them so understanding any disk space limitations is important.
• For custom training in Vertex AI, one can use an interactive shell to inspect the container where the training code is running, as described here. Is something like this possible for a custom prediction container? I have not found anything in the docs.
• For custom training, one can use a private IP for custom training as described here. Again, I have not found anything similar for custom prediction in the docs, is it possible?

If you know of any other possible limitations, please post.

Answer 1

1. we don't specify a disk size, so default to 100GB
2. I'm not aware of this right now. But if it's a custom container, you could just run it locally or on GKE for debugging purpose.
3. are you looking for this? https://cloud.google.com/vertex-ai/docs/predictions/using-private-endpoints