CODEWITHSUNDEEP
pysparkaws-lambdaaws-glue
RK.
RK.

Reputation: 607

Can we trigger AWS Lambda function from aws Glue PySpark job?

Currently i'm able to run Glue PySpark job, but is this possible to call a lambda function from Glue this job ? Using below code from my PySpark Glue job i'm calling lambda function.

lambda_client = boto3.client('lambda', region_name='us-west-2')
response = lambda_client.invoke(FunctionName='test-lambda')

Error:

botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the Invoke operation: User: arn:aws:sts::208244724522:assumed-role/AWSGlueServiceRoleDefault/GlueJobRunnerSession is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:us-west-2:208244724522:function:hw-test

But I added proper lambda roles to my Glue iam role, still getting above error. Any specific role need to add ?

Thanks.

Upvotes: 1

Views: 2873

Answers (1)

statut
statut

Reputation: 909

To invoke AWS Lambda you can use the following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowToExampleFunction",
            "Effect": "Allow",
            "Action": "lambda:InvokeFunction",
            "Resource": "arn:aws:lambda:<region>:<123456789012>:function:<example_function>"
        }
    ]
}

Your roles are not suitable for Lambda invocations as

AWSLambdaBasicExecutionRole – Grants permissions only for the Amazon CloudWatch Logs actions to write logs. You can use this policy if your Lambda function does not access any other AWS resources except writing logs.

AWSLambdaVPCAccessExecutionRole – Grants permissions for Amazon Elastic Compute Cloud (Amazon EC2) actions to manage elastic network interfaces (ENIs).

Please see documentation here about these roles.

Upvotes: 2

Related Questions