CODEWITHSUNDEEP
mongodbmeteor
Graham P
Graham P

Reputation: 2272

Weirdness when mixing native Mongo and Meteor collections

I've got a server method that loops through a Mongo collection directly (i.e. not using Meteor collections):

var db = MongoInternals.defaultRemoteCollectionDriver().mongo.db;

db.collection("dataRecords").find({}).each(...)

Inside that each() callback, I do a read using Meteor collections on an unrelated collection:

ReportDefs.findOne(...);

When the read operation is executed, that path of execution is paused and the next record begins processing. This keeps happening. At the same, execution resumes on some of the paused paths, but this overlaps with the processing of new records. The result is a big mess.

So it looks like findOne() is no longer synchronous inside the each() callback?

Does anyone know how I can safely mix native Mongo and Meteor collections? I'm trying to avoid the overhead of Meteor collections when processing large numbers of records.

Upvotes: 1

Views: 167

Answers (2)

Graham P
Graham P

Reputation: 2272

I eventually figured this out. If you do

cursor.each()

and wait() anywhere in the callback, the next each() callback can start, so you end up processing records in parallel, and the server method can return before everything is done.

The solution is to use cursor.nextObject() in the callback to process the next record only when you're done processing the current. You should wrap it in a process.nextTick(), otherwise you keep going deeper into the stack.

Upvotes: 0

Tomasz Lenarcik
Tomasz Lenarcik

Reputation: 4880

Try wrapping up your "each callback" with Meteor.bindEnvironment. The problem comes from the fact that meteor code strongly depends on the existence of a "current fibre" (look here for more details). This is why the synchronous behavior of routines like findOne is even possible. So you need to provide a running Fiber, but Meteor.bindEnvironment will already take care of it.

It's interesting though, since Meteor always used to warn the users about this issue, e.g. "meteor code must always run within a fiber". Didn't you get any messages of this type?

Upvotes: 2

Related Questions