Reputation: 135

Triggering AWS Glue Workflow through Lambda function

I am new to AWS GLUE and trying to trigger Glue workflow using the Lambda function.

I am using the attribute boto3.client('glue') but I am getting an error saying :

Glue' object has no attribute start_workflow_run

Here is the piece of code that I am trying to run:

import json
import boto3
def lambda_handler(event, context):
client = boto3.client('glue')
client.start_workflow_run(Name = 'Workflow_New', Arguments = {})

Is there any other method by what I can achieve what I am trying to do?

Upvotes: 4

Answers (6)

Ishita Mahajan

Reputation: 1

Try using:

import json
import boto3

def lambda_handler(event, context):
    glueClient = boto3.client('glue', region_name='us-west-2')
    response = glueClient.start_workflow_run(Name=Workflow_name)

Also, I think you might want to add error handling around the response as well!

Upvotes: 0

Prerna Bharti

Reputation: 41

Please try using the below code snippet:

import boto3

glueClient = boto3.client('glue')

response = glueClient.start_workflow_run(Name = 'wf_name')

You can use also this documentation: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.start_workflow_run

Upvotes: 1

semaphore

Reputation: 99

this works to invoke a glue workflow from Lambda (python):

import json
import boto3
def lambda_handler(event, context):

    # add your region_name
    glue = boto3.client(service_name='glue', region_name='eu-west-2') 
    
    # only 'Name' parameter
    workflow_run_id = glue.start_workflow_run(Name = 'Your_Workflow')

    print(f'workflow_run_id: {workflow_run_id}')

https://docs.aws.amazon.com/glue/latest/dg/glue-dg.pdf

Upvotes: 1

Yuva

Reputation: 3153

Please refer to this SO on how to call AWS Glue from a lambda, with code snippet.

How to Trigger Glue ETL Pyspark job through S3 Events or AWS Lambda?

import boto3
print('Loading function')

def lambda_handler(event, context):
    source_bucket = event['Records'][0]['s3']['bucket']['name']
    s3 = boto3.client('s3')
    glue = boto3.client('glue')
    gluejobname = "YOUR GLUE JOB NAME"

    try:
        runId = glue.start_job_run(JobName=gluejobname)
        status = glue.get_job_run(JobName=gluejobname, RunId=runId['JobRunId'])
        print("Job Status : ", status['JobRun']['JobRunState'])
    except Exception as e:
        print(e)
        print('Error getting object {} from bucket {}. Make sure they exist '
              'and your bucket is in the same region as this '
              'function.'.format(source_bucket, source_bucket))
    raise e

Thanks

Yuva

Upvotes: 1