CODEWITHSUNDEEP
pythonamazon-web-servicesdockeramazon-s3amazon-sagemaker
Raghav Arora
Raghav Arora

Reputation: 51

Download data from S3 bucket onto SageMaker inference container

I deployed a model to a SageMaker endpoint for inference. My input data is quite large and I would like to send its S3 URI to the endpoint instead, so that I can download it onto the deployed Docker container. Unfortunately, when I try using the SageMaker SDK to download the data, I get this error:

Read-only file system: '/opt/ml/model/'

I would really appreciate if someone could help me solve this issue.

Upvotes: 2

Views: 3487

Answers (1)

Raghav Arora
Raghav Arora

Reputation: 51

The SageMaker Inference instance has a directory called /temp/ which is writeable, and can be used for non-persistent storage. I used the S3Downloader utility in the SageMaker SDK to download data to this directory. For instance:

from sagemaker.s3 import S3Downloader
from sagemaker.session import Session

sagemaker_session = Session()

def download_data_from_s3(s3_uri):
    S3Downloader.download(s3_uri=s3_uri,
                          local_path='/tmp/',
                          sagemaker_session=sagemaker_session)

Upvotes: 3

Related Questions