CODEWITHSUNDEEP
javaazureasynchronousgoogle-cloud-platformcloud
Constantine
Constantine

Reputation: 381

Async Tasks in cloud environment

I am trying to find a way to run async tasks in cloud for java application.

For example, we are running our application in GCP and there is a way to run Deferred Tasks through Task Queues. But there is a problem with serialization of tasks. I am not always able to make entire hierarchy serializable.

Also there is a possibility to use ExecutorService, but there is a downside to that: it cannot run background threads for instances with auto scaling.

Main purpose of my task is to separate request thread from long running tasks. Also it would be good to run those tasks on separate machines. I guess there should be some way to achieve my goal, but I am not able to see it yet.

Upvotes: 1

Views: 350

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 75940

When you have Java object that you want to defer, there is no secret, you have to serialize them. This can take several form

  • Implement serializable interface and use Cloud Task
  • Store the object somewhere (firestore, Google Storage) and only pass to the task the reference (documentId of Firestore, BlobId of GCS) for reloading the data and process them
  • Publish a message into PubSub and trigger your endpoint with the data embedded in the PubSub message

In all these possibilities, you have to serialize the data (in binary, in JSON, in text,...)

Anyway, for "long running job", all depend of the "long" and the resources required. Today, Cloud Run is a great candidate for this with 15 minutes of timeout (soon 4x more) and 2vCPU/2Gb of memory (soon more). If that fit your requirement, I strongly encourage you to have a look on it.

  • It's serverless
  • It's portable
  • You perform the run on another VM

Upvotes: 2

Related Questions