CODEWITHSUNDEEP
amazon-web-servicesamazon-swf
Lloyd Banks
Lloyd Banks

Reputation: 36699

AWS SWF Simple Workflow - Best Way to Keep Activity Worker Scripts Running?

The maximum amount of time the pollForActivityTask method stays open polling for requests is 60 seconds. I am currently scheduling a cron job every minute to call my activity worker file so that my activity worker machine is constantly polling for jobs.

Is this the correct way to have continuous queue coverage?

Upvotes: 0

Views: 1029

Answers (2)

Lloyd Banks
Lloyd Banks

Reputation: 36699

I ended using a solution where I had another script file that is called by a cron job every minute. This file checks whether an activity worker is already running in the background (if so, I assume a workflow execution is already being processed on the current server).

If no activity worker is there, then the previous long poll has completed and we launch the activity worker script again. If there is an activity worker already present, then the previous poll found a workflow execution and started processing so we refrain from launching another activity worker.

Upvotes: 0

mkobit
mkobit

Reputation: 47319

The way that the Java Flow SDK does it and the way that you create an ActivityWorker, give it a tasklist, domain, activity implementations, and a few other settings. You set both the setPollThreadCount and setTaskExecutorSize. The polling threads long poll and then hand over work to the executor threads to avoid blocking further polling. You call start on the ActivityWorker to boot it up and when wanting to shutdown the workers, you can call one of the shutdown methods (usually best to call shutdownAndAwaitTermination).

Essentially your workers are long lived and need to deal with a few factors:

  1. New versions of Activities
  2. Various tasklists
  3. Scaling independently on tasklist, activity implementations, workflow workers, host sizes, etc.
  4. Handle error cases and deal with polling
  5. Handle shutdowns (in case of deployments and new versions)

Upvotes: 2

Related Questions