parquetjs on AWS Lambda function not working

Question

I'm trying to use the @dsnp/parquetjs library to read a parquet file in AWS Lambda

The Lambda is throwing this error:

2023-09-21T05:49:13.435Z    undefined   ERROR   Uncaught Exception  
{
    "errorType": "Error",
    "errorMessage": "ENOENT: no such file or directory, open '/var/task/wasm_brotli_nodejs_bg.wasm'",
    "code": "ENOENT",
    "errno": -2,
    "syscall": "open",
    "path": "/var/task/wasm_brotli_nodejs_bg.wasm",
    "stack": [
        "Error: ENOENT: no such file or directory, open '/var/task/wasm_brotli_nodejs_bg.wasm'",
        "    at Object.openSync (node:fs:590:3)",
        "    at Object.readFileSync (node:fs:458:35)",
        "    at node_modules/wasm-brotli/wasm_brotli_nodejs_bg.js (/var/task/index.js:18200:31)",
        "    at __require (/var/task/index.js:10:50)",
        "    at node_modules/wasm-brotli/wasm_brotli_nodejs.js (/var/task/index.js:18292:12)",
        "    at __require (/var/task/index.js:10:50)",
        "    at node_modules/@dsnp/parquetjs/dist/lib/compression.js (/var/task/index.js:18307:25)",
        "    at __require (/var/task/index.js:10:50)",
        "    at node_modules/@dsnp/parquetjs/dist/lib/schema.js (/var/task/index.js:18635:44)",
        "    at __require (/var/task/index.js:10:50)"
    ]
}

In the simplest code I'm saving the file to Lambda's file system (/tmp/) and just opening a buffer like so:

  const file = fs.readFileSync(localFilePath);
  console.log('file:', file);
  const reader = await parquet.ParquetReader.openBuffer(file);

The parquet file defiantly exists (I can confirm with the console.log), but Lambda throws that error.

Any ideas on what this could be? I don't see any reference to brotli anywhere in my code.

Answer 1

Try to import the package like this:

import parquetjs from "@dsnp/parquetjs/dist/browser/parquet.esm"

Docs