CODEWITHSUNDEEP
meteor
by chase
by chase

Reputation: 3

In meteor how to keep server side computing after returning first partial data to the client

A noob to Meteor and even node.js. I'm thinking of a simple web app, and really like the simplicity of meteor. But I do need some advice on the best usage.

The basic flow of the app is:

As the google api has limits on how many data can be returned in one call, and the analysis may be time consuming, I'm wondering can the app return partial data to the user first, while keeping fetching and computing. Once some more data is ready, the app push the additional data to the user.

I don't know what's the best practice to continue data fetching and computing in the server side after responding partial result to the client. I guess I can use setInterval, but the delay value is hard to select (as it's variant), and I want the next chunk of data fetching/processing will wait for the current one to complete. Sorry if it's a naive question, but your suggestion is highly appreciated.

Upvotes: 0

Views: 160

Answers (2)

Philip Nuzhnyy
Philip Nuzhnyy

Reputation: 4740

Alternatively you could outsource some of the tasks to background jobs (ie crons). synced-cron package worked really nice for us : http://atmospherejs.com/package/synced-cron

Upvotes: 0

richsilv
richsilv

Reputation: 8013

If I understand your question correctly, then the Meteor side of things takes care of itself. If you're inserting the results into the MongoDB on the server and your client is subscribing to the right data (which might be in their profile, or else a separate document(s) with their _id attached), then the client will be updated automatically whenever new data is inserted or existing data is amended. You only have to worry about getting things right on the server.

So the question is really what the most efficient way to fetch, process and periodically store the data is, and without knowing what the processing involves it's pretty difficult to answer.

Is the data being fetched in a single get? If not, then just queue the analysis by associated get and store after each batch is complete. Is the analysis being done in a loop? If so, just store the data after every x cycles.

Basically, if the processing is sufficiently arduous that it takes so long you need to store partial results, then there must be a logical way of working out when the best time to store those results is. And once you do, Meteor will take care of publishing the changes to the client automatically - the client doesn't need to know how long it's going to have to wait for an update.

Note that you can actually do this without even storing the results in the server database, using the low-level publications API, although it would be a bit more complicated.

Upvotes: 2

Related Questions