AWS Step Functions - is there such a solution for Google Cloud Platform?

Question

At the moment I am investigating the possibility and the proper way of migrating complex web applications from AWS to GCP. There is actually no issues with mapping general compute and networking services from one provider to another, but I wonder if GCP has a service similar to AWS Step Functions? I've already taken a look at Google Dataflow and Google Cloud Tasks. The second one seems to be something like that, but I am not sure if it's the optimal solution.

So the question is what service from google provides same functionality as AWS Step Functions? And if there is no such - then combination of which services would you recommend to achieve effective orchestration of distributed tasks (primarily cloud functions). Thanks!

Answer 1

2021 Update

As Brian de Alwis noted below, since this answer was written Cloud Workflows is now generally available and is functionally similar to Step Functions.

---

2019 Answer

As far as I'm aware there's nothing specifically like Step Functions, but I have two strategies for creating these types of micro-service systems on Google Cloud.

Strategy 1: Cloud Run/Cloud Functions with Pub/Sub

Here I'd create microservices using Cloud Run or Cloud Functions and subscribe these functions to Pub/Sub topics. That means that when Function A executes and completes it's work, it publishes a message to a specific topic with a data packet that any function subscribed to it will receive and execute.

For example you could create two topics named FunctionASuccess and FunctionAError and create two separate functions that subscribe to one or the other and handle the success and error use cases.

Strategy 2: Firebase Functions with Firestore/Realtime Database

Similarly to above I create Firebase Functions that watch for changes in Firestore or in the RTDB.

So Function A executes and completes its task, it saves a document to the FunctionAResults collection in Firestore or RTDB. Functions that are subscribed to changes in the FunctionAResults collection are then executed and take it to the next step.

---

They both work reliably so I have no preference, but I typically go with the 2nd strategy if I'm utilizing other Firebase services.

Answer 2

Cloud Workflows was announced at Cloud Next On Air 2020.

Answer 3

You're looking for Cloud Composer. It's based on the open-source library Apache Airflow which allows you to define and orchestrate workflows in a similar way to step functions.