CODEWITHSUNDEEP
pythonamazon-web-servicesboto3amazon-sagemaker
pelos
pelos

Reputation: 1876

boto3 how to get the logstream form a sagemaker transform job?

i am able to crete the job and it fail, using boto3

import boto3
session = boto3.session.Session()


client = session.client('sagemaker')
descibe = client.describe_transform_job(TransformJobName="my_transform_job_name")

in the ui i can see the button to go to the logs, i can use boto3 to retrive the logs if hardcode the group name and the log-stream.

but how can i get the Log stream from the batch transfrom job? shouldnt be a field with logstream or something like that in the ".describe_transform_job"?

Upvotes: 1

Views: 919

Answers (1)

pelos
pelos

Reputation: 1876

sagemaker doesnt provide a direct way to do it, the way to do it, is to also use the log client.

  1. get the log streams corresponding to your batchtransform_job

    client_logs = boto3.client('logs')
log_groups = 
client_logs.describe_log_streams(logGroupName="the_log_group_name",                                                   logStreamNamePrefix=transform_job_name)
log_streams_names= []
for i in log_groups["logStreams"]:
    log_streams_names.append(i["logStreamName"])

and this will give a list of "project_name/virtualMachine_id" that is the machines that your code was run depending on how many instances you set.

  1. After you can run for each of the log_streams

     for i_stream_name in log_streams_names:
     client_logs.get_log_events("the_stream_log_name", "the_log_group_name")

now you can loop and print the lines of the log stream event =)

Upvotes: 1

Related Questions