CODEWITHSUNDEEP
linuxamazon-web-servicesamazon-sagemakerpapermill
Led
Led

Reputation: 517

aws auto-stop-idle does not detect papermill

I am using papermill to parametrize jupyter notebook deployed on AWS Sagemaker. I also used this lifestyle configuration that will auto shutdown if there are no running/idle notebooks. Unfortunately, it does not detect the Papermill process and continues to shutdown after reaching the specified idle time. What do I need to do to keep Sagemaker alive until the completion of Papermill

Upvotes: 1

Views: 160

Answers (1)

Gili Nachum
Gili Nachum

Reputation: 5578

You could edit the idleness detection script to account for papermill processes.

Alternatively, if you have async jobs, which you can formulate as python code, you could use SageMaker processing jobs to execute them, which will not depend on your notebook instance being up.

Upvotes: 1

Related Questions