CODEWITHSUNDEEP
meteor
crapthings
crapthings

Reputation: 2485

Does cursor.observe slow down server performance?

i've searched and found

_suppress_initial: true

but it doesnt work with 0.54

i want to observe some collection like an Orders Collection.

if i have huge orders and when new order added that i want to use observe to update another collection.

I didnt put observe into Meteor.publish what if i don't stop that observe, does that slow down server what if i keep it observering all the time during the server running ?

if Meteor.isServer

    obOrders = Orders.find({}).observe # when server restart does this slow down performance ?

        _suppress_initial: true # doesnt work

        added: (order) ->

            console.log order # still add exist documents

            if Date.now() - order.timestamp < 500
                console.log order # update another one

or should i limit Orders.find {}, limit: 50 and sort by timestamp to observe latest documents ?

To put observeon server Meteor.startup or Meteor.publish what's the different between that two condition?

if i put it into Meteor.startup does that mean i do a singleton observering ?

Upvotes: 5

Views: 1144

Answers (1)

Geoff
Geoff

Reputation: 4642

Right now, observe has to keep the entire results of the query permanently in memory. So if you call Orders.observe({}), then the server will keep an entire copy of the Orders collection in memory for as long as the observe is running. This is because, behind the scenes, observe works by diffing old query results to new query results when a potential change is detected. This is to be sure that the results from observe are always 100% correct even if there are race conditions as a result of several processes write to the database simultaneously.

Thus, if you query for a limited number of documents, like the five most recent orders, or the orders placed in the last 5 minutes, it will dramatically reduce RAM usage (and possibly CPU usage.) But if you do this you will have to be careful to make sure that you don't miss any documents. For example, if you are observing the documents that were inserted in the last 5 minutes, but your server goes down for 10 minutes, then you might not get added messages for some orders. Or, if you are observing the 5 most recent documents, and then another node inserts 1000 documents at once, you might only receive added message for the most recent 5 messages (because observe doesn't guarantee that you will observe every intermediate state, just that your add/remove/change messages will eventually bring you up to date with the current state.)

As for where you would start such an observe: If you do it from Meteor.startup on the server, it will always be running, but there will only be one copy of it running (for now -- in the future Meteor will start multiple server processes and you will need to update your code.) If you do it from a publish handler, then it will only run (and only consume resources) while there is at least one subscription. Recent Meteor releases will de-duplicate observes that are called on exactly the same query, so if you have 1000 subscriptions to a publish handler that calls Orders.find({}).observe({ ... }), it shouldn't consume that much more resources than if there were only one. (Remember to stop each observe when the client unsubscribes.)

Depending on what you're doing, it might be easier to schedule a task when each Order is created, rather than use added. For example, you could have a method createOrder that inserts an order, and also adds an item to an ordersToProcess collection, and then separately have code that pulls items out of ordersToProcess collection and processes them one at a time and then removes them. Or you could just do your processing directly from the createOrder method if it doesn't take too long.

Upvotes: 6

Related Questions