CODEWITHSUNDEEP
amazon-web-servicesdockeraws-lambda
Viplove
Viplove

Reputation: 151

AWS lambda read-only file system error, using docker image to store ML model

I am using a docker container image on lambda to run my ML model. My lambda function has a S3 trigger to fetch images. I am trying to run my lambda function but I am getting this error. Can someone please help me?

error screenshot

PS - now i am aware /tmp is the only writable directory in lambda but how to solve this with that?

Upvotes: 7

Views: 7214

Answers (3)

Raphael Setin
Raphael Setin

Reputation: 865

As others have mentioned, /tmp is the only writable directory in any AWS Lambda environments, either using containers or not.

Having said that, you should move your entire library (during the lambda runtime -- during container image build time doesn't work) to that directory -- such that everything remains connected within the library -- and then reference your new library directory in the library path environment for Lambda: LD_LIBRARY_PATH

Referencing your new library directory in the library path environment for Lambda should be done because Lambda looks at the /opt/ directory by default; and since you just moved your library to /tmp, you should also update LD_LIBRARY_PATH to contain that location. This can be done in the Dockerfile:

# Set the LD_LIBRARY_PATH
ENV LD_LIBRARY_PATH="/opt/my-lib-folder/:$LD_LIBRARY_PATH"

or during Lambda runtime:

os.environ['LD_LIBRARY_PATH'] = '/tmp/my-lib-folder:' + os.environ['LD_LIBRARY_PATH']

def lambda_handler(event, context):
   # your code ...

If there are still problems, it may be related to linking problems of your library, or that you didn't update your LD_LIBRARY_PATH correctly.

EDIT: As pointed out by @rok, you cannot move your libraries during the container image build time, because the /tmp folder will be erased by AWS automatically.

Upvotes: 13

MisterSmith
MisterSmith

Reputation: 3644

That error seems to be it trying to create its config/cache folders and failing. Essentially you need to specify the MPLCONFIGDIR environment variable to your container and set the value to a directory inside /tmp that is writable.

Upvotes: 1

Mark B
Mark B

Reputation: 200960

The file system in a Lambda environment is read-only except for the /tmp directory.

Upvotes: 4

Related Questions