CODEWITHSUNDEEP
pythonjupyter-notebookaws-glue
kunshabba
kunshabba

Reputation: 51

AWS Glue Notebook update job timeout

I have a AWS glue notebook that should run for about 15 minutes. The current job timeout value in job details is set to 10 minutes. I'm not able to directly change this value through the UI since it's a glue notebook. Instead, it says I should use %magic commands, but I don't see a magic command for job timeout, only idle_timeout. How can I update the job timeout value so my job can run for longer than 10 minutes?

Upvotes: 0

Views: 157

Answers (1)

Cristián Vargas Acevedo
Cristián Vargas Acevedo

Reputation: 712

hi this is how you can set timeout parameter , the magic %idle_timeout is only for interactive sessions

glue_client = boto3.client('glue')

# Nombre del trabajo de Glue que deseas actualizar
job_name = "my_job_name"
new_timeout = 120
try:
# Recuperar la configuración actual del trabajo
response = glue_client.get_job(JobName=job_name)
job_details = response['Job']  # Extraer los detalles del trabajo

# Construir la actualización con los valores existentes
response = glue_client.update_job(
    JobName=job_name,
    JobUpdate={
        'Role': job_details['Role'],  # Mantener el rol actual
        # 'Description': job_details['Description'],   
        # 'CreatedOn': job_details['CreatedOn'], ##bad  
        # 'LastModifiedOn': job_details['LastModifiedOn'],  #bad
        'ExecutionProperty': job_details['ExecutionProperty'],   
        # 'Connections': job_details['Connections'],   
        'GlueVersion': job_details['GlueVersion']
        'MaxRetries': job_details['MaxRetries']   , 
        'Command': job_details['Command'],  # Mantener el comando actual
        'Timeout': new_timeout,  # Actualizar el Timeout
        # 'DefaultArguments': job_details['DefaultArguments']   ,         
        # 'AllocatedCapacity': job_details['AllocatedCapacity']   ,            
        # 'WorkerType': job_details['WorkerType']   ,            
        # 'MaxCapacity': job_details['MaxCapacity']   ,            
        # 'NumberOfWorkers': job_details['NumberOfWorkers']   ,               
        # 'ExecutionClass': job_details['ExecutionClass']   ,
    }
)
    print(f"Timeout del job '{job_name}' actualizado a {new_timeout} minutos.")
except glue_client.exceptions.EntityNotFoundException:
    print(f"El trabajo '{job_name}' no existe. Verifica el nombre.")
except Exception as e:
    print(f"Error al actualizar el Timeout: {e}")

Upvotes: 0

Related Questions