CODEWITHSUNDEEP
javascriptnode.jsamazon-web-servicescachingaws-lambda
Kosmetika
Kosmetika

Reputation: 21304

AWS Lambda /tmp folder is not shared between executions

I'm trying to temporary cache some data for AWS lambda function executions to omit HTTP request that this lambda does every time right now and optimise its' speed.

In "viewer request" I have the following logic (example is pseudo-code) -

exports.handler = async (event) => {
   // check /tmp folder for data
   const cachedData = await getDataFromCache();

   if (cachedData) {
     return cachedData;
   }

   const data = await getDataFromApi();
   
   // save data for 2 mins in /tmp for re-use in next executions
   await saveDataInCache(data);

   return data;
}

As you can see, I'm trying to save the data in /tmp folder for a few mins in order to re-use it in next executions and reduce the number of API requests.

During "viewer request" lambda execution, when I cache data inside /tmp folder, I immediately see that data is available in the logs -

async function saveDataInCache(data = {}) {
  // save data in tmp folder
  try {
    await fs.outputJson('/tmp/cache.json', {
      data,
      updatedAt: Date.now()
    });
  } catch (err) {
    console.log(`page data save cache error ${err}`);
  }

  // check that data is immediately available
  let fileData = {};
  try {
    fileData = await fs.readJson('/tmp/cache.json');
  } catch (err) {
    console.log(`page data read cache error ${err}`);
  }

  console.log(`check immediate value ${JSON.stringify(fileData)}`);
}

However every time when "viewer request" tries to read the data before saving, it always returns error - ENOENT: no such file or directory, open '/tmp/cache.json' -

async function getDataFromCache() {
  let fileData = {};

  try {
    fileData = await fs.readJson('/tmp/cache.json');
  } catch (err) {
    console.log(`page data read cache error ${err}`);
  }

  if (isEmpty(fileData)) {
    return undefined;
  }

  const { data = {}, updatedAt = 0 } = fileData;
  const maxCacheTime = updatedAt + DATA_TTL;
  const currentTime = Date.now();
  const isCacheExpired = currentTime > maxCacheTime;

  if (!isCacheExpired) {
    return data;
  }

  return undefined;
}

Does it mean that /tmp folder content is not shared between executions? Thanks!

Upvotes: 2

Views: 5952

Answers (2)

Mrk Fldig
Mrk Fldig

Reputation: 4486

Short answer is as Maurice said, /tmp is unique to each container, so if you're lucky you'll get the same container back, but it's not guaranteed. After the container is disposed of all files are gone.

Use EFS, S3, or Lambda layers if you need real persistent storage between Lambda's.

That being said what's the use case?

Cache.json updated during each execution

Then use EFS/s3(bit slower than EFS) OR a NoSQL database if you're already using one, another collection isn't going to change the world and will be super fast.

Static file cache.json

Either bundle this up with your lambda code or put it in a layer!

Upvotes: 3

Maurice
Maurice

Reputation: 13093

The /tmp directory is private to each Lambda instance/execution context, it's not shared among all Lambda instances. That means if your request is handled by a different execution context, the data won't be present there.

Unfortunately you can't really control how these execution contexts are created and deleted, that's something Lambda handles for you in the background.

Upvotes: 8

Related Questions