Hosting multiple models for multiple datasets in aws sagemaker

Question

I read that there is a way to train and host multiple models using a single endpoint for a single dataset in AWS Sagemaker. But I have 2 different datasets in S3 and have to train a model for each dataset. Can these 2 different models be hosted using a single endpoint?

Answer 1

Yes, this is called a multi-model endpoint. You can use a large number of models on the same endpoint. They get loaded and unloaded dynamically as needed, and you simply have to pass the model name in your prediction request.

Here are some resources:

• Blog post + example : https://aws.amazon.com/blogs/machine-learning/save-on-inference-costs-by-using-amazon-sagemaker-multi-model-endpoints/

• Video explaining model deployment scenarios on SageMaker: https://youtu.be/dT8jmdF-ZWw