CODEWITHSUNDEEP
javascriptnode.jsmemorymemory-leaks
dkoch
dkoch

Reputation: 474

Where is the memory leak / simple callbacks?

I have simple code, but it has memory leak, which I cant find. After some minutes memory usage increasing from 75MB to 180MB and going on. Running on Node v0.12.7

function tock() {
  extractKeywords('2x Hannets® Akku 3500 mAh für Neato Modell 945-0005, 945-0006, XV Signature Pro, XV Signature, XV Essential, XV-12, XV-15, XV-11', function(err) {
    if (err) {
      console.log(err)
    }

    setImmediate(tock)
  })
}

// run
tock();

function extractKeywords(title, cb) {
  title = title.replace(/[^\w\-_]+/gi, ' ').replace(/\s+/gi, ' ')

  redis.get('keywords_count', function(err, keywords) {
    if(!err) {
      lastKeywordCount = keywords = parseInt(keywords)
    }

    // var keywords = keyword_extractor.extract(title, {
    //   language:"german",
    //   remove_digits: true,
    //   return_changed_case: true,
    //   remove_duplicates: true
    // })
    var keywords = ['keyword1', 'keyword2', 'keyword3', 'keyword4', 'keyword5'].filter(function(keyword) {
      if (keyword.length < 5)
        return false
      return true
    }).map(function(k) {
      return [lastKeywordCount, k]
    }).reduce(function(a,b) {
      return a.concat(b)
    })

    keywords.unshift('keywords')

    cb()

    // var keywords = ['keywords', 0, 'test', 1, 'test1', 1, 'test2', 1, 'test3', 1, 'test4']

    // redis.ZADD(keywords, function(err, count){
    //   if (err){
    //     return cb(err)
    //   }

    //   redis.INCRBY('keywords_count', count, cb)
    // })
  })
}

I tried node-inspector but I cant figure out where the leak comes from. If I remove filter/map/reduce/unshift, it seems like the leak is gone or leaks only less memory.

Upvotes: 4

Views: 452

Answers (1)

Yuri Zarubin
Yuri Zarubin

Reputation: 11677

It may not actually be a memory leak, it's difficult to tell with Node sometimes. Node.js uses Google's V8 JavaScript engine, which implements the garbage collector. The GC algorithm that the V8 uses, will attempt to take up as much memory as it can, and is very lazy when it comes to cleaning up unused memory. As such, it is typical that Node processes will continue increasing in memory usage until a certain point where the GC thinks it's better to free memory. Note that Node processes have a hard memory limit of 1.4 GB.

There is a way to force the GC to free memory, using the --expose-gc flag:

node --expose-gc yourscript.js

Then add this code somewhere at the beginning of your application. This will force garbage collection every 30 seconds.

setInterval(function() {
  global.gc();
  console.log('GC done')
}, 30000);

Afterwards, run your code for a good amount of time. If your memory is still consistently increasing over time, then it's most likely a legit memory leak.

Upvotes: 4

Related Questions