CODEWITHSUNDEEP
google-app-enginecloudgoogle-cloud-platformgoogle-compute-engine
nkxandroid
nkxandroid

Reputation: 779

How to push CPU-heavy tasks from Google App Engine to Google Compute Engine VM instance?

I have CPU-heavy simulation tasks I would like to execute in the following manner using Google App Engine and Google Compute Engine:

  1. Simulation task data is submitted through Google App Engine, e.g. by executing an API call /runsimulation?param1=...param2=...
  2. Inside the GAE servlet, the simulation task is set up and its computational requirements are determined, i.e. what type of compute engine machine is necessary
  3. The GAE servlet starts the required VM instance
  4. The GAE servlet somehow transfers the simulation task to the VM instance where it is processed
  5. The GAE servlet is notified when the VM instance has finished. Ideally, it can also receive progress updates during the simulation and display them to the user.
  6. Finally, the GAE servlet receives the simulation result from the VM instance and hands it over to the user. The VM instance is subsequently shut down.

I am having my difficulties with 4 and 5. How can I manage to "push" a computation task to a Compute Engine VM instance from a GAE servlet? And how can GAE and VM instance communicate afterwards?

(While I am experienced with GAE, I am totally new to Compute Engine and have my difficulties entirely grasping the Compute Engine approach to problems like mine)

Thanks in advance!

Upvotes: 1

Views: 426

Answers (1)

Arne S
Arne S

Reputation: 1016

If you just want to spin up the instance to perform one task the easiest would probably be to provide the task in the form of a startup script as explained here

The GCE instance could report back the status of the work by issuing http requests to GAE.

Alternatively you could communicate between GAE and GCE using sub/pub. In that case - and unless you assume that the user has the browser open showing a progress page and that this page pulls GAE for updates - you would probably need to set up a cron job on GAE to check for finished tasks and act accordingly.

Upvotes: 1

Related Questions