CODEWITHSUNDEEP
javascriptnode.jsmongodbmeteor
user2402616
user2402616

Reputation: 1563

Meteor server restarts itself upon slow requests

I have a Meteor app that is performing some calls that are currently hanging. I'm processing a lot of items in a loop that is then upserting to server-side Mongo. (I think this is done asynchronously) I understand the upserting in a loop is not good .

This whole functionality seems to make the app hang for a while. I'm even noticing sock.js and websocket error out in the console. I think this is all due to DDP, async Mongo upserts, and the slow requests. Here's some pseduocode to what I'm talking about

for (1..A Lot of records) {
    //Is this async?
    Collection.upsert(record)
}

Eventually this function will complete. However, I'll notice that Meteor "Restarts" (I think this is true because I see Accounts.onLogin being called again. It's almost like the client refreshes after the slow request has actually finished. This results in something that appears like an infinite loop.

My question is why the app is "restarting". Is this due to something in the framework and how it handles slow requests? I.e. does it queue up all bad requests and then eventually retry them automatically?

Upvotes: 1

Views: 138

Answers (1)

Mikkel
Mikkel

Reputation: 7777

I am not sure about what exactly is going on here, but it sounds like the client isn't able to reach the server while it is "busy", and then the client connection over DDP times out, and ends up with a client refresh. The server process probably doesn't restart.

One technique for improving this is to implement a queue in your database. One piece of code detects there are a bunch of database upserts to do, so it records the information in a table which is used as a queue.

You set up a cron job (using eg npm module node-cron) that looks for things in the queue on a regular basis - when it finds an unprocessed record, it does the upsert work needed, and then either updates a status value in the queue record to 'done', or simply deletes it from the queue. You can decide how many records to process at a time to minimise interruptions.

Another approach is to do the processing in another node process on your server, basically like a worker process. If this process is busy, it is not going to impact your front end. The same queueing technique can be used to make sure this doesn't get bogged down either.

You lose a little reactivity this way, but given it's some kind of bulk process, that shouldn't matter.

Upvotes: 2

Related Questions