CODEWITHSUNDEEP
node.jsmongodbstream
user11810894
user11810894

Reputation:

How to stream documents from a collection in Mongodb

I have this:

d.db(dbName)
     .collection(dbCollName)
     .find()
     .sort({_id: 1})
     .limit(5000).stream()

this expression returns this:

enter image description here

basically all I am trying to do is instead of this:

   d.db(dbName)
         .collection(dbCollName)
         .find()
         .sort({_id: 1})
         .limit(5000).toArray().then(console.log);

I instead want to stream the documents one-by-one etc, if that's possible.

Upvotes: 2

Views: 2596

Answers (1)

ambianBeing
ambianBeing

Reputation: 3529

Yes mongodb cursor that is returned by db.collection().find() does have a stream() api to stream the documents:

const stream = db
  .collection("dbCollName")
  .find()
  .stream();

stream.on("data", function(data) {
  //get streamed documents one by one
  const currentDoc = data;
});

stream.on("error", function(error) {
  console.error("STREAM ERROR::", error.stack);
});

stream.on("end", function() {
  console.info("Streaming docs finished");
  client.close(); //new MongoClient(url, { useNewUrlParser: true });
});

Sometime back wrote a demo test for stream with mongodb node driver cursor to clone a collection for a fair amount of dataset. some more info

Upvotes: 2

Related Questions