CODEWITHSUNDEEP
node.jsmemory-leaks
Nrgyzer
Nrgyzer

Reputation: 1075

Memory usage of Node.js application increases when writing files

I'm connected to a p2p network and receiving messages every some seconds. The messages can have a size of 10 chars but also 5 MB. I need to log all messages I receive by writing them to the file system. At the moment I'm using fs.writeFileSync() but unfortunately, the memory increases the longer my application is running:

In this case, my application is running for 15 hours and uses already 3.7Gi. The function I use for writing my logs is the following:

let counter = 0

class Message {
    constructor(data) {
        this.msgId = data.readInt32LE()
        const length = data.readInt32LE()
        // this.data = JSON.parse(data.read(<length>))
    }
    
    // some other methods

    save() {
        if (!fs.existsSync(`/data/${this.msgId}`)) {        
            const data = Object.assign(this.data, {
              index: counter++
            })
            console.log(counter)
            fs.writeFileSync(`/data/${this.msgId}`, JSON.stringify(data))
          }
    }
}

Whenever I receive a message over p2p, I'm reading the msgId and the data it contains within the constructor. After doing some other business logic on the "Message" object, I finally calling the save() function. The "memory leak" is definitely inside the save() function because when I leave the save() function empty, my application runs for days without cosnuming more than 200MB memory. So does anyone know if fs.writeFilesync() has a potential memory leak or what might cause the memory leak?

Upvotes: 0

Views: 1286

Answers (1)

Mark D
Mark D

Reputation: 26

I am having the same issue while using node 15.8.0 on windows 10.

I replaced fs.writeFileSync(files[i], data, { encoding: 'utf8' }); with

let fd = 0;
try {
   fd = fs.openSync( files[i], 'w', 0o666  );  
   let writen = fs.writeSync( fd, data, 0, 'utf8' );
} catch ( e ) {
} finally {
   if ( fd )
     fs.closeSync(fd);
}

This resolved the issue with expanding heap.

Upvotes: 1

Related Questions