CODEWITHSUNDEEP
node.jszlibarchiverjs
brendandevwork
brendandevwork

Reputation: 1

Node zlib crazy memory consumption

I'm writing a script that appends JSON to a write stream using Node's archiver package. archiver then tarballs and gzips the write stream to a local file on disk.

When I run this script however, the process' memory consumption can get up to 2GB! Without using the gzip: true flag in archiver (that is, just using archiver to append to stream and tar), the script only uses 300MB of memory.

I worked around this by spawning a child process to gzip the tarball, but I'd love to know two things:

  1. Why is archiver and by extension, zlib, moving the whole tarball to memory for compression?
  2. Is there a way to work around this without exec-ing to the shell?

Thanks!

Upvotes: 0

Views: 214

Answers (0)

Related Questions