Applying SAGA pattern in situations where immediate feedback to user is required

Question

Imagine there is an app where a user has a wallet which they can top it up with cash, cash it out or make purchases from an external system, when the user creates a new purchase order, we first deduct the amount from the user’s wallet. Then send an API call to the external API saying the user purchased these items and we get a response back from the merchant on whether the purchase was successful or not. If the purchase was not successful, we would refund the amount to the user’s wallet on our system. However, one key note here is that the merchant purchase API endpoint could return error responses that are domain errors (user is not registered, user has been deactivated, purchase is less than minimum allowed amount or above maximum amount) and the user gets an immediate confirmation response on whether the transaction was successful or not, and if not, we show the user the failure reason we got from the external API

I’d like to apply saga to the flow above but there are some challenges

1. Let’s say we’re going to be using a message broker (Kafka, rabbitmq) for async saga flow, how do we return a response to the user on whether the transaction was successful or not? The async transaction could fail for any reason, and if it fails it might take a while to process retries or even rollbacks in the background.

2. And even if we were able to let’s say notify the front-end/user of the result using something like webhooks where we push data to the client. What happens on timeouts or technical failures? Since the flow is async, it could take either a second or an hour to finish. Meanwhile what should the user see? If we show a timeout error, the user could retry the request and end up with 2 requests in pending state that will be processed later on but the user’s intention was only to make one.

I cannot show the user a successful message like “Purchase created” then notify them later on for two reasons:

1. The external API returns domain errors a lot of the time. And their response is immediate. So it won’t make sense for the user to see this response message then immediately get a notification about the failure
2. The user must be able to see the error message returned by the external API

How do we solve this? The main reason behind attempting to solve it with saga is to ensure consistency and retry on failure, but given that, how do we handle user interaction?

Answer 1

This is how I would solve this through temporal.io open source project:

1. Synchronously (waiting for completion) execute a workflow when a user creates the purchase order.
2. Workflow deducts the purchase amount from the user's wallet
3. Workflow calls the external API
4. If the API call completes without failure complete the workflow. This unblocks the synchronous call from (1) and shows the status to the user.
5. If the API call fails start (without waiting for the result) another workflow that implements the rollback.
6. Fail the original workflow. This returns the failure to the caller at (1). This allows showing the error to the user.
7. The rollback workflow executes the rollback logic as long as needed.

Here is the implementation of the above logic using Java SDK. Other supported SDKs are Go, Typescript/Javascript, Python, PHP.

public class PurchaseWorkflowImpl implements PurchaseWorkflow {

  private final ActivityOptions options =
      ActivityOptions.newBuilder().setStartToCloseTimeout(Duration.ofSeconds(10)).build();
  private final Activities activities = Workflow.newActivityStub(Activities.class, options);

  @Override
  public void purchase(String accountId, Money amount, List<Item> items) {
    WalletUpdate walletUpdate = activities.deductFromWallet(accountId, amount);
    try {
      activities.notifyItemsPurchased(accountId, items);
    } catch (ActivityFailure e) {
      // Create stub used to start a child workflow.
      // ABANDON tells child to keep running after the parent completion.
      RollbackWalletUpdate rollback =
          Workflow.newChildWorkflowStub(
              RollbackWalletUpdate.class,
              ChildWorkflowOptions.newBuilder()
                  .setParentClosePolicy(ParentClosePolicy.PARENT_CLOSE_POLICY_ABANDON)
                  .build());
      // Start rollbackWalletUpdate child workflow without blocking.
      Async.procedure(rollback::rollbackWalletUpdate, walletUpdate);
      // Wait for the child to start.
      Workflow.getWorkflowExecution(rollback).get();
      // Fail workflow.
      throw e;
    }
  }
}

The code that synchronously executes workflow

    PurchaseWorkflow purchaseWorkflow =
        workflowClient.newWorkflowStub(PurchaseWorkflow.class, options);
    // Blocks until workflow completion.
    purchaseWorkflow.purchase(accountId, items);

Note that Temporal ensures that the code of the workflow keeps running as if nothing happened in the presence of various types of failures including process crashes. So all the fault tolerance aspects are taken care of automatically.