CODEWITHSUNDEEP
node.jsamazon-web-servicesaws-lambdafs
David Faizulaev
David Faizulaev

Reputation: 5751

AWS Lambda with Node - saving files into Lambda's file system

I need to save files I get from S3 into a Lambda's file system and I wanted to know if I can do that simply using fs.writeFileSync ?

Or do I have to still use the context function as described here: How to Write and Read files to Lambda-AWS with Node.js

(tried to find newer examples, but could not).

What is the recommended method?

Please advise.

Upvotes: 2

Views: 6120

Answers (2)

jarmod
jarmod

Reputation: 78842

Yes, you can use the typical fs functions to read/write from local disk, but be aware that writing is limited to the /tmp directory and the default max diskspace available to your Lambda function in that location is 512 MB. Also note that files written there may persist to the next (warm) Lambda invocation.

If you want to simply download an object from S3 to the local disk (assuming it will fit in the available diskspace) then you can combine AWS SDK methods and Node.js streaming to stream the content to disk.

Also, it's worth noting that, depending on your app, you may be able to process the entire S3 object in RAM via streaming, without any need to actually persist to disk. This is helpful if your object size is over 512MB.

Update: as of March 2022, you can now configure Lambda functions with more ephemeral storage in /tmp, up to 10 GB. You get 512 MB included with your Lambda function invocation and are charged for the additional configured storage above 512 MB.

If you need to persist very large files, consider using Elastic File System.

Upvotes: 6

zbauman
zbauman

Reputation: 225

Lambda does not allow access to the local file system. It is mean to be an ephemeral environment. They allow access to the /tmp folder, but only for a maximum of 512MB. If you want to have storage along with your function, you will need to implement AWS S3 or AWS EFS.

Here's an article from AWS explaining this.

Here's the docs on adding storage to Lambda.

Upvotes: 0

Related Questions