CODEWITHSUNDEEP
google-app-engine
Ido Ran
Ido Ran

Reputation: 11384

Long running work in GAE

I'm writing an application that read messages from twitter on GAE. The application should read messages all the time and react to them. Because I'm currently testing an idea I don't want to use backends, which are the natural choice for such need. I'm using a cron job that run every minute and servlet which handle the job by running for as long as 50 seconds continually reading messages.

  1. I've notice that GAE open up to 3 instances to handle only this trafic, is that bad?
  2. Backends are the right tool for such job, right?
  3. Is there better way of doing it without backends?

Thank you.

Upvotes: 0

Views: 235

Answers (2)

Peter Knego
Peter Knego

Reputation: 80340

  1. Spinning up of instances is controlled by application settings. If your request takes long time to complete, then it might trigger spinning up of new instances - depending on said settings.

  2. Fetching data can be very effectively done with Task Queue. You do not need Backends for this. Backends are useful when you need to keep data in memory between requests, effectivelly saving on Datastore requests.

  3. As said - look into Task Queue for fetch tasks: How long does a data fetch task take? How often do you need them to perform?

Upvotes: 2

Takashi Matsuo
Takashi Matsuo

Reputation: 3436

I've notice that GAE open up to 3 instances to handle only this trafic, is that bad?

I don't think it's bad, because thanks to those 3 instances, your cron keeps working fine.

Cron execution can be delayed few seconds, spinning up the instance could also take some additional seconds, and besides that, after the 50 seconds, presumably your app runs some code, which can also take some time.

If you have some evidence that shows the instances ran previous cron is in available state when a particular cron request is invoked, please share it here.

Backends are the right tool for such job, right?

Absolutely.

Is there better way of doing it without backends?

It's somewhat difficult to answer unless knowing what is 'better' for you. Is there any problem with the current strategy?

Upvotes: 1

Related Questions