CODEWITHSUNDEEP
pythonamazon-web-servicesdockeraws-lambdadockerfile
Rnne Msly
Rnne Msly

Reputation: 57

Access S3 event notification from within python in a Lambda container

As a cloud developer, I am currently working on a AWS architecture which include both AWS S3 and Lambda services.

My goal here is to be able to catch the S3 notification event from a python function in a Lambda container.

The workflow is the following :

  1. A file is dropped in a bucket-A
  2. It triggers a Lambda built with a container image stored in ECR (AWS container registry)

Typically, for a classic lambda function, what you define as your handler must be prototyped as def lambda_handler(event, context); to properly receive the S3 event notification.

In the example found here, we can see that using

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "app.handler" ]

where app is the filename (app.py) and handler the function within this file can allow us to reach the code. Note: app.py is in the working directory defined by WORKDIR from Dockerfile.

Directory Structure (EDIT)

The route_serializer.py is the python file that needs to be reached from the Dockerfile. It contains a function called lambda_handler that will handle the S3 notifications and process the newly arrived file. This is the exact structure present within the container when running the Lambda.

project/
├── aws/
│   ├── __init__.py
│   ├── lambdas/
│   │   ├── batchDisaggregator/
│   │   │   ├── __init__.py
│   │   │   └── route_serializer.py <-- contains lambda_handler(evt,ctx);
│   │   └── __init__.py

What I tried

So I would like to know if there's a way I can call the handler function (def lambda_handler(event, context);) from the CMD override available in Lambda, and catch the S3 notification within the event function argument.

Upvotes: 1

Views: 661

Answers (1)

Rnne Msly
Rnne Msly

Reputation: 57

Ok, I figured it out by myself. For everyone with the same issues, here's some leads that might help you aswell

In my case, I was missing 2 things :

  • My custom-based image used to run the container had the awslambdaric package to make it compliant with lambda. But I needed to run my function with the following CMD : /path/to/python -m awslambdaric batchDisaggregator.route_serializer to allow the runtime interface to call my function.

  • Function code and all the dependecies should be stored under the path specified in the LAMBDA_TASK_ROOT environment variable. You can define it within the Dockerfile using ENV LAMBDA_TASK_ROOT=/var/task

Feel free to comment if you need more information about how I solved this

Upvotes: 1

Related Questions