CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-lambdaaws-event-bridge
maxmoore14
maxmoore14

Reputation: 877

How to trigger a Lambda on S3 file upload

I can't seem to get my lambda to trigger as I would expect. In AWS EventBridge, I created a rule with the following custom event pattern:

{
  "source": ["aws.s3"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": [
      "CopyObject",
      "CompleteMultipartUpload",
      "PutObject"
    ],
    "requestParameters": {
      "bucketName": ["my-bucket"],
      "key": [{"prefix": "folder1/folder2/"}]
    }
  }
}

but when I upload a file to the "directory" the rule does not trigger the lambda.

Can someone tell me where I've gone wrong?

Upvotes: 3

Views: 11854

Answers (5)

Geocoder
Geocoder

Reputation: 141

As of 2021-11-29, Amazon Event Bridge now supports Amazon S3 Event Notifications without having to enable CloudTrial.

In the properties of the S3 bucket you want to monitor, you have to enable Amazon EventBridge.

enter image description here

This will allow you to send messages to EventBridge using Rules. enter image description here

In your eventbridge rule, you can configure lambda as a "Target". I made a step-by-step tutorial on how to configure eventbridge in AWS if you want to follow along: https://youtu.be/k-jEuNb_KBM

Upvotes: 2

maxmoore14
maxmoore14

Reputation: 877

Thanks to all for the help and suggestions - all good resources should someone come across this post in the future. Turns out in my case @Balu Vyamajala was correct in the comments - I had incorrectly configured CloudTrail.

Upvotes: 1

Jens
Jens

Reputation: 21500

As mentioned in my comment to your question before, you might not have the proper permissions to allow EventBridge to Invoke your Lambda.

You can add the following Resource-based policy to your Lambda:

{
  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "AllowExecutionFromEventBridge",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "<lambda-arn>",
      "Condition": {
        "ArnLike": {
          "AWS:SourceArn": "<eventbridge-rule-arn>"
        }
      }
    }
  ]
}

You need to replace <lambda-arn> and <eventbridge-rule-arn> with the respective values.

You can read more about resource-based policy here: Using resource-based policies for AWS Lambda.

If you use tools like Terraform, you can use the following snippet:

resource "aws_lambda_permission" "example" {
  statement_id  = "AllowExecutionFromCloudWatch"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.example.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.example.arn
}

You need to replace function_name and source_arn here as well, with your specific references.

Upvotes: 0

Siva Sumanth
Siva Sumanth

Reputation: 662

You can use both S3 events or cloud watch event rule to invoke the lambda function for any event on S3.

  • Below Image shows how to configure cloud watch event rule for s3 putObject operation.
  • Make sure you enabled cloud trail for data events in the respective region.
  • Make sure you create a rule for the specific bucket.
  • if you mention is for all the buckets. You will get unnecessary invocations, as cloud watch event rules works on cloud trail, where it will store all the logs to s3. enter image description here

Upvotes: 1

Siva Sumanth
Siva Sumanth

Reputation: 662

You can use event notification in S3. It has direct integration with Lambda, SNS, SQS

  • Goto properties tab in S3

  • Navigate to Event notification. click on create event notification. enter image description here

  • Then add the event name.

  • If you want to restrict the event to a specific folder or file type, you can fill in prefix or suffix fields or if you want it for entire bucket leave those blank. enter image description here

  • Then select a list of events. like put, post.enter image description here

  • Then choose the destination and click on save. You will be notified. enter image description here

Upvotes: 2

Related Questions