CODEWITHSUNDEEP
node.jsredisstream
Hoang Doanh
Hoang Doanh

Reputation: 99

node.js module for reading redis key as stream

I have problem in reading keys in Redis on Node.js and them save data to mongodb. Because the number of keys is huge ( 1.3 million), this causes JavaScript heap out of memory.

Could anyone suggest me some Node.js module to read Redis keys as stream or some other solution to this problem, so I could avoid this issue.

Thank you very much !

Upvotes: 0

Views: 2874

Answers (1)

Tuan Anh Tran
Tuan Anh Tran

Reputation: 7237

ioredis (https://github.com/luin/ioredis) does support streams.

Example with SCAN from their readme.

var redis = new Redis();
// Create a readable stream (object mode)
var stream = redis.scanStream();
var keys = [];
stream.on('data', function (resultKeys) {
  // `resultKeys` is an array of strings representing key names
  for (var i = 0; i < resultKeys.length; i++) {
    keys.push(resultKeys[i]);
  }
});
stream.on('end', function () {
  console.log('done with the keys: ', keys);
});

Upvotes: 2

Related Questions