Understanding Event Payload Structure and Processing in a .NET Lambda Function Triggered by S3 Events via AWS EventBridge

Question

I'm currently working on an AWS serverless application using .NET, where I'm configuring a Lambda function to be triggered by S3 events routed through AWS EventBridge. I want to have a single Lambda function that can handle all types of S3 events, like object creation, modification, deletion, etc. In this setup, I'm specifically seeking guidance on understanding the event payload structure passed by EventBridge to my .NET Lambda function and how to process it effectively.

Here are my primary areas of inquiry:

Event Payload Structure: What is the structure of the event payload passed by EventBridge to my .NET Lambda function when triggered by S3 events? I'm interested in understanding the key properties and fields included in the event payload.

Deserialization and Processing: How can I deserialize and process the event payload within my .NET Lambda function to perform actions based on the S3 event that triggered it? I'm looking for insights into efficiently handling the event payload to extract relevant information and take appropriate actions based on the event type (e.g., object creation, deletion).

If anyone has experience with configuring EventBridge to trigger .NET Lambda functions with S3 events or can provide guidance on handling the event payload within a .NET Lambda function, I would greatly appreciate it. Any code examples, resources, or best practices related to event payload processing in a .NET Lambda function would be immensely helpful.

Thank you in advance for your assistance!

Answer 1

The boilerplate code created by the S3 Event sub-type of the C# AWS Lambda projects in Visual Studio should work for any type of S3 event:

using Amazon.Lambda.Core;
using Amazon.Lambda.S3Events;
using Amazon.S3;
using Amazon.S3.Util;

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

namespace AWSLambdaS3;

public class Function
{
    IAmazonS3 S3Client { get; set; }

    /// <summary>
    /// Default constructor. This constructor is used by Lambda to construct the instance. When invoked in a Lambda environment
    /// the AWS credentials will come from the IAM role associated with the function and the AWS region will be set to the
    /// region the Lambda function is executed in.
    /// </summary>
    public Function()
    {
        S3Client = new AmazonS3Client();
    }

    /// <summary>
    /// Constructs an instance with a preconfigured S3 client. This can be used for testing outside of the Lambda environment.
    /// </summary>
    /// <param name="s3Client">The service client to access Amazon S3.</param>
    public Function(IAmazonS3 s3Client)
    {
        this.S3Client = s3Client;
    }

    /// <summary>
    /// This method is called for every Lambda invocation. This method takes in an S3 event object and can be used 
    /// to respond to S3 notifications.
    /// </summary>
    /// <param name="evnt">The event for the Lambda function handler to process.</param>
    /// <param name="context">The ILambdaContext that provides methods for logging and describing the Lambda environment.</param>
    /// <returns></returns>
    public async Task FunctionHandler(S3Event evnt, ILambdaContext context)
    {
        var eventRecords = evnt.Records ?? new List<S3Event.S3EventNotificationRecord>();
        foreach (var record in eventRecords)
        {
            var s3Event = record.S3;
            if (s3Event == null)
            {
                continue;
            }

            try
            {
                var response = await this.S3Client.GetObjectMetadataAsync(s3Event.Bucket.Name, s3Event.Object.Key);
                context.Logger.LogInformation(response.Headers.ContentType);
            }
            catch (Exception e)
            {
                context.Logger.LogError($"Error getting object {s3Event.Object.Key} from bucket {s3Event.Bucket.Name}. Make sure they exist and your bucket is in the same region as this function.");
                context.Logger.LogError(e.Message);
                context.Logger.LogError(e.StackTrace);
                throw;
            }
        }
    }
}

Depending on your exact need, you may not actually need the S3 client, nor the filtering on events that have the S3 block populated.

Answer 2

Here is a blog Event-driven .NET applications with AWS Lambda and Amazon EventBridge that can do a lot of help to you.

Answering your question from the blog itself.

To handle S3 EventBridge events, you need to add the Amazon.Lambda.CloudWatchEvents NuGet package to the project.

Run dotnet add package Amazon.Lambda.CloudWatchEvents

If you are wondering why the package is called Amazon.Lambda.CloudWatchEvents, it is because CloudWatch Events is the predecessor of EventBridge.

The Lambda function will look like this.

using Amazon.Lambda.Core;
using Amazon.Lambda.CloudWatchEvents.S3Events;
using System.Text.Json;

[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

namespace EventBridgeLambdaAndS3Example;

public class Function
{
    public void FunctionHandler(S3ObjectCreateEvent s3ObjectCreateEvent, ILambdaContext context)
    {
        context.Logger.LogLine("S3 Object Create Event Received");
        context.Logger.LogLine(JsonSerializer.Serialize(s3ObjectCreateEvent));
    }
}