CODEWITHSUNDEEP
node.jsmemory-leaksgoogle-cloud-platformamazon-sqs
Alex L.
Alex L.

Reputation: 101

Node.js AWS-SDK SQS memory leak

I've wrote a websocket module in Node.js. Running it under Debian Jessie on a VM in a Google cloud. It runs as a service via PM2. The service gets a request, sends a response and also sends some data to AWS SQS(query service).

I have a custom class for the sending messages to the queue, and it has memory leak in it:

var AWS = require("aws-sdk");
var logger = require("./Logger");

AWS.config.loadFromPath("/home/admin/.aws/cred.json");

function QueueService() {
    this.sqs = new AWS.SQS();
    this.queue = "https://sqs.eu-central-1.amazonaws.com/4864251684/MyQueue";
}

QueueService.prototype.sendItemToStatistics = function (message, reqJson, wsConnection, queue) {
    try {
        logger.log("silly", "QueueUrl", queue)

        this.sqs.sendMessage({
            MessageBody: message, 
            QueueUrl: queue,
            DelaySeconds: 0
        }, 
        function (err, data) {
            if (err) logger.log("error", "Error, while sending report to statistics:", err.stack)
            else logger.log("debug", "Submitted to SQS")
        });

    } catch (e) {
        logger.log("error", "Failed to send a statistics report, stacktrace:", e.stack)
    }
}

module.exports = new QueueService();

This is the problematic part - sending items to the queue:

this.sqs.sendMessage({
                MessageBody: message, 
                QueueUrl: queue,
                DelaySeconds: 0
            }, 
            function (err, data) {
                if (err) logger.log("error", "Error, while sending report to statistics:", err.stack)
                else logger.log("debug", "Submitted to SQS")
            });

With ~100 RPS, the service RAM gets bloated to ~900mb in about 4-5 minutes. I then kill and respawn the process to keep it responsive. If I comment this out, memory leak stops, service stays at around 60-70 mb RAM under same conditions.

I assume SDK is fine and there is something wrong with my implementation. Didn't find any particular info on this issue.

Anyone experienced this?

Upvotes: 2

Views: 1984

Answers (1)

Alex L.
Alex L.

Reputation: 101

As usual, I find the answear minutes later: Add this:

require('http').globalAgent.maxSockets = require('https').globalAgent.maxSockets = 100

At the first line, before the rest of the code. This forces node to rehash used sockets.

It turns out default value for maxSockets is unlimited, so it just keeps opening new sockets instead of reusing the old ones.

As suggested by mhart here

Upvotes: 4

Related Questions