CODEWITHSUNDEEP
javascriptnode.js
Álvaro
Álvaro

Reputation: 2588

Removing properties from array of objects, memory impact

I am processing a heavy array of objects on a node app, and at some point I want to remove two properties from all objects in the array.

At the same time I am measuring the node memory impact on rss (Residente Set Size).

What I am finding is that just the act of deleting them consumes a lot of memory.

Example, it's actually a lot bigger and with lots of objects. File size of json is 200MB.

[
  {
   keep: 'foo',
   prop1: 'remove',
   prop2: 'remove'
  },...
]

This consumes the most from 500MB goes to 1000MB

const clean = original.map((obj) => {
  delete obj.prop1
  delete obj.prop2
  return obj
})

This also consumes a lot, also around 1000MB

original.forEach((obj) => {
  delete obj.prop1
  delete obj.prop2
})

This comsumes the least, goes to around 650MB

const clean = original.map(({ prop1, prop2, ...obj }) => obj)

But if I do not delete them at all then it does not consume anymore than the original 500MB. What is going on? Should not removing properties make the memory lighter?

Upvotes: 2

Views: 431

Answers (2)

skoll
skoll

Reputation: 2452

Node uses the V8 engine, and V8 engine has two different internal representations of objects. A so called fast mode (default) and a slow mode (aka dictionary mode). When you delete a property from an object, the object gets converted to slow mode. What is significant in this case, is that the slow mode also uses more memory.

If possible, use obj.prop1 = null instead.

Read more: https://v8.dev/blog/fast-properties

Upvotes: 0

Randy Casburn
Randy Casburn

Reputation: 14165

You can use either of the .map() calls you provided (they are faster than .forEach()).

When you've completed the deletion, simply set the original variable to null. This will allow the JS engine to mark this variable for Garbage Collection and it will be removed from memory. You won't see the result immediately as we have no idea when GC will run and cannot control it. But the memory will be reclaimed very quickly.

Like this:

const clean = original.map((obj) => {
  delete obj.prop1
  delete obj.prop2
  return obj
})
original = null;

const clean = original.map(({ prop1, prop2, ...obj }) => obj);
original = null;

The issue here is that we as web devs don't have any control over memory consumption other than what I have proposed. Internally the V8 engine decides how things are put together in memory and all memory management issues are relegated to it.

Upvotes: 3

Related Questions