CODEWITHSUNDEEP
amazon-web-servicesaws-lambdajupyter-notebookamazon-sagemakeraws-emr-studio
user22
user22

Reputation: 153

How to automate jupyter notebook execution on aws?

I got a task to complete where I need to automate Jupyter notebook execution on AWS. I'm totally new to AWS environment so don't have any idea how to do it efficiently. Things I need to do are the following -

  1. Need REST API(s) to start and stop Jupyter notebook execution on AWS.
  2. Need to send parameters to the notebook while calling using API.

What are the AWS components I need, to perform the above task?

import boto3,time
 
emr = boto3.client(
    'emr',
    region_name='us-west-1'
)
 
 
 
start_resp = emr.start_notebook_execution(
    EditorId='e-40AC8ZO6EGGCPJ4DLO48KGGGI',
    RelativePath='boto3_demo.ipynb',
    ExecutionEngine={'Id':'j-1HYZS6JQKV11Q'},
    ServiceRole='EMR_Notebooks_DefaultRole'
)
 
execution_id = start_resp["NotebookExecutionId"]
print(execution_id)
print("\n")
 
 
describe_response = emr.describe_notebook_execution(NotebookExecutionId=execution_id)
 
print(describe_response)
print("\n")
 
 
 
list_response = emr.list_notebook_executions()
print("Existing notebook executions:\n")
for execution in list_response['NotebookExecutions']:
    print(execution)
    print("\n")
 
 
 
print("Sleeping for 5 sec...")
time.sleep(5)
 
print("Stop execution " + execution_id)
emr.stop_notebook_execution(NotebookExecutionId=execution_id)
describe_response = emr.describe_notebook_execution(NotebookExecutionId=execution_id)
print(describe_response)
print("\n")

Upvotes: 1

Views: 903

Answers (1)

Kirit Thadaka
Kirit Thadaka

Reputation: 537

Here is an experimental repository that creates a Jupyter Extension in SageMaker Studio to automate running your notebooks. It has been designed for trial use and there's no guarantee of SageMaker support.

https://github.com/aws-samples/sagemaker-run-notebook

I work at AWS and my opinions are my own.

Upvotes: 1

Related Questions