How to abstract the DTF framework in the application layer?

Question

I have a question regarding clean architecture and durable task framework. But first, let me show you by example what we can do with DTF. DTF enables us to run workflows/orchestrations of individual task in the background. Here is an example:

public class EncodeVideoOrchestration : TaskOrchestration<string, string>
{
    public override async Task<string> RunTask(OrchestrationContext context, string input)
    {
        string encodedUrl = await context.ScheduleTask<string>(typeof (EncodeActivity), input);
        await context.ScheduleTask<object>(typeof (EmailActivity), input);
        return encodedUrl;
    }
}

The TaskOrchestration wires together individual tasks into a workflow. Here is how you define the tasks:

public class EncodeActivity : TaskActivity<string, string>
{
    protected override string Execute(TaskContext context, string input)
    {
        Console.WriteLine("Encoding video " + input);
        // TODO : actually encode the video to a destination
        return "http://<azurebloblocation>/encoded_video.avi";
    }
}

public class EmailActivity : TaskActivity<string, object>
{
    protected override object Execute(TaskContext context, string input)
    {
        // TODO : actually send email to user
        return null;
    }
}

Pretty straight forward, right? Then you create a worker in Program.cs and register all the tasks and orchestrations:

TaskHubWorker hubWorker = new TaskHubWorker("myvideohub", "connectionDetails")
    .AddTaskOrchestrations(typeof (EncodeVideoOrchestration))
    .AddTaskActivities(typeof (EncodeActivity), typeof (EmailActivity))
    .Start();

Using the DTF client you can actually trigger an orchestration:

TaskHubClient client = new TaskHubClient("myvideohub", "connectionDetails");
client.CreateOrchestrationInstance(typeof (EncodeVideoOrchestration), "http://<azurebloblocation>/MyVideo.mpg");

DTF handles all the magic in the background and can use different storage solutions such as service bus or even mssql.

Say our application is organized into folders like this:

• Domain
• Application
• Infrastructure
• UI

In tasks we run application logic / use cases. But the DTF framework itself is infrastructure, right? If so, how would an abstraction of the DTF framework look like in the application layer? Is it even possible to make the application layer unaware of the DTF?

Answer 1

Durable Task Framework has already done all the abstractions for you. TaskActivity is your abstraction:

public abstract class TaskActivity<TInput, TResult> : AsyncTaskActivity<TInput, TResult>
{
    protected TaskActivity();

    protected abstract TResult Execute(TaskContext context, TInput input);
    protected override Task<TResult> ExecuteAsync(TaskContext context, TInput input);
}

You can work with TaskActivity type in your Application Layer. You don't care about its implementation. The implementation of TaskActivity goes to lower layers (probably Infrastructure Layer, but some tasks might be more suitable to be defined as a Domain Service, if they contain domain logic)

---

If you want, you can also group the task activities, for example you can define a base class for Email Activity:

Domain Layer Service (Abstraction)

public abstract class EmailActivityBase : TaskActivity<string, object>
{
    public string From { get; set; }

    public string To { get; set; }

    public string Body { get; set; }
}

This is your abstraction of an Email Activity. You Application Layer is only aware of EmailActivityBase class.

Infrastructure Layer Implementation

The implementation of this class goes to Infrastructure Layer:

Production email implementation

public class EmailActivity : EmailActivityBase
{
    protected override object Execute(TaskContext context, string input)
    {
        // TODO : actually send email to user
        return null;
    }
}

Test email implementation

public class MockEmailActivity : EmailActivityBase
{
    protected override object Execute(TaskContext context, string input)
    {
        // TODO : create a file in local storage instead of sending an email
        return null;
    }
}

Where to Put Task Orchestration Code?

Depending on your application, this may change. For example, if you are using AWS you can use AWS lambda for orchestration, if you are using Windows Azure, you can use Azure Automation or you can even create a separate Windows service to execute the tasks (obviously the Windows service will have dependency on your application). Again this really depends on your application but it may not be a bad idea to put these house keeping jobs in a separate module.

Answer 2

You could wrap the activities in a library that returns simple Tasks, and might mix long-running activities with short-running ones. Something like

public class BusinessContext
{
    OrchestrationContext context;
    public BusinessContext(OrchestrationContext context)
    {
        this.context = context;
    }

    public async Task<int> SendGreeting(string user)
    {
        return await context.ScheduleTask<string>(typeof(SendGreetingTask), user);
    }
    public async Task<string> GetUser()
    {
        return await context.ScheduleTask<string>(typeof(GetUserTask));
    }

}

Then the orchestration is a bit cleaner

    public override async Task<string> RunTask(OrchestrationContext context, string input)
    {
        //string user = await context.ScheduleTask<string>(typeof(GetUserTask));
        //string greeting = await context.ScheduleTask<string>(typeof(SendGreetingTask), user);
        //return greeting;

        var bc = new BusinessContext(context);
        string user = await bc.GetUser();
        string greeting = await bc.SendGreeting(user);
        return greeting;
    }

Answer 3

You have to fit your implementation with the Ubiquitous language. In the specific example: Who and when does encoding happen? Whichever entity or service (the client) does the encoding will simply call an IEncode.encode interface that'll take care of the "details" involved in invoking a DTF.

Yes, the definition for DTF is in the Infrastructure and it should be treated like everything else in the infrastructure like Logging or Notifications. That is: The functionality should be put behind an interface that can be injected into the Domain and used by its Domain Clients.

Answer 4

I think your question is not really just a single question regarding the code but a request for the whole concept of how to make that main program "unaware" of the specific DTF library you going to use.

Well, it involves several areas of functionality you will need to use in order accomplish that. I added a diagram for how the architecture should look like to achieve what you ask for, however I didn't focus on the syntax there since the question is about architecture and not code itself as I understood it, so treat it as a pseudo code - it is just to deliver the concept.

The key idea is you will have to read the path or name of the DLL you wish to load from a configuration file (such as app.config) but to do that you will need to learn how to create custom configuration elements in a configuration file. You can read about those in the links:

https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/

https://learn.microsoft.com/en-us/dotnet/api/system.configuration.configuration?view=dotnet-plat-ext-6.0

Next you need to dynamically load the assembly, you can read about how to load assemblies dynamically here https://learn.microsoft.com/en-us/dotnet/framework/app-domains/how-to-load-assemblies-into-an-application-domain

Once you passed that, remember that the DLL you are loading is still something you need to implement and it needs to be aware of the specific DTF Library you wish to reference, however it also implement an interface well known in your application as well.

So basically you will have an interface describing the abstraction your program need from a DTF library (any DTF library) and your Proxy DLL which will be loaded at runtime will act as mediator between that interface which describe that abstraction and the actual implementation of the specific DTF library.

And so, per your questions:

how would an abstraction of the DTF framework look like in the application layer?

Look at the diagram I provided.

Is it even possible to make the application layer unaware of the DTF?

Yes, like in any language that can support plugins/extensions/proxies

Answer 5

In regards to Clean Architecture approach, if you want to get rid of DTF in the Application layer, you can do following (original repo uses MediatR, so I did as well)

• implement TaskActivity as query/command and put it in Application layer

using MediatR;

public class EncodeVideoQuery : IRequest<string>
{
    // TODO: ctor

    public string Url { get; set; }
}

public class EncodeHandler : IRequestHandler<EncodeVideoQuery, string>
{
    public async Task<string> Handle(EncodeVideoQuery input, CancellationToken cancel)
    {
        Console.WriteLine("Encoding video " + input);
        // TODO : actually encode the video to a destination
        return "http://<azurebloblocation>/encoded_video.avi";
    }
}

public class EmailCommand 
{
    public string UserEmail { get; set; }
}

public class EmailCommandHandler : IRequestHandler<EmailCommand>
{
    public async Task<Unit> Handle(EmailCommand input, CancellationToken cancel)
    {
        // TODO : actually send email to user
        return Unit.Value;
    }
}

• implement actual DTF classes (I looked up that they support async) and put them into a "UI" layer. There's no UI, but technically it's a console application.

using MediatR;

public class EncodeActivity : TaskActivity<string, string>
{
    private readonly ISender mediator;

    public EncodeActivity(ISender mediator)
    {
        this.mediator = mediator;
    }

    protected override Task<string> ExecuteAsync(TaskContext context, string input)
    {
        // Perhaps no ability to pass a CancellationToken
        return mediator.Send(new EncodeVideoQuery(input));
    }
}