CODEWITHSUNDEEP
google-cloud-platformgoogle-ai-platformgoogle-cloud-vertex-ai
racerX
racerX

Reputation: 1092

Vertex AI custom container batch prediction

I have created a custom container for prediction and successfully uploaded the model to Vertex AI. I was also able to deploy the model to an endpoint and successfully request predictions from the endpoint. Within the custom container code, I use the parameters field as described here, which I then supply later on when making an online prediction request. My questions are regarding requesting batch predictions from a custom container for prediction.

  1. I cannot find any documentation that describes what happens when I request a batch prediction. Say, for example, I use the my_model.batch_predict function from the Python SDK and set the instances_format to "csv" and provide the gcs_source. Now, I have setup my custom container to expect prediction requests at /predict as described in this documentation. Does Vertex AI make a POST request to this path, converting the cvs data into the appropriate POST body?

  2. How do I specify the parameters field for batch prediction as I did for online prediction?

Upvotes: 6

Views: 2091

Answers (1)

Shawn
Shawn

Reputation: 1593

  1. Yes vertex AI makes a POST request your custom containers in batch prediction.

  2. No, there is no way for batch prediction to pass a parameter since we don't know which column is "parameter". We put everything into "instances".

Upvotes: 4

Related Questions