CODEWITHSUNDEEP
amazon-sagemaker
Shiva Kishore
Shiva Kishore

Reputation: 1701

Continuous Training in Sagemaker

I am trying out Amazon Sagemaker, I haven't figured out how we can have Continuous training.
For example if i have a CSV file in s3 and I want to train each time the CSV file is updated.

I know we can go again to the notebook and re-run the whole notebook to make this happen.
But i am looking for an automated way, with some python scripts or using a lambda function with s3 events etc

Upvotes: 4

Views: 1689

Answers (2)

robperc
robperc

Reputation: 21

There are a couple examples for how to accomplish this in the aws-samples GitHub.

The serverless-sagemaker-orchestration example sounds most similar to the use case you are describing. This example walks you through how to continuously train a SageMaker linear regression model for housing price predictions on new CSV data that is added daily to a S3 bucket using the built-in LinearLearner algorithm, orchestrated with Amazon CloudWatch Events, AWS Step Functions, and AWS Lambda.

There is also the similar aws-sagemaker-build example but it might be more difficult to follow currently if you are looking for detailed instructions.

Hope this helps!

Upvotes: 2

Fahri
Fahri

Reputation: 472

You can use boto3 sdk for python to start training on lambda then you need to trigger the lambda when csv is update.

http://boto3.readthedocs.io/en/latest/reference/services/sagemaker.html

Example python code

https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-train-model-create-training-job.html

Addition: You dont need to use lambda you just start/cronjob the python script any kind of instance which has python and aws sdk in it.

Upvotes: 2

Related Questions