Proper method of creating, not awaiting, and ensuring completion of, Tasks

Question

Preface: I don't have a good understanding of the underlying implementation of tasks in C#, only their usage. Apologies for anything I butcher below:

I'm unable to find a good answer to the question of "How can I start a task but not await it?" in C#. More specifically, how can I guarantee that the task completes even if the context of the task is finalized/destroyed?

_ = someFunctionAsync(); is satisfactory for launching and forgetting about a task, but what if the parent is transient? What if the task cannot complete before the parent task? This is a frequent occurrence in controller methods, and tasks written in the fashion _ = someFunctionAsync(); are subject to cancellation.

Example Code:

        [HttpGet]
        public IActionResult Get()
        {
            _ = DoSomethingAsync();
            return StatusCode(204);
        }

In order to combat this cancellation, I created a (fairly stupid, imo) static class to hold onto the tasks so that they have time to complete, but it does not work, as the tasks are cancelled when the parent controller is destroyed:

    public static class IncompleteTaskManager
    {
        private static ConcurrentBag<Task> _incompleteTasks = new();

        private static event EventHandler<Task>? _onTaskCompleted;

        public static void AddTask(Task t)
        {
            _onTaskCompleted += (sender, task) =>
            {
                _incompleteTasks = new ConcurrentBag<Task>(_incompleteTasks.Where(task => task != t));
            };

            _incompleteTasks.Add(CreateTaskWithRemovalEvent(t));
        }
        
        private static async Task CreateTaskWithRemovalEvent(Task t)
        {
            await t;
            _onTaskCompleted?.Invoke(null, t);
        }
    }

Plus, this seems convoluted and feels like a bad solution to a simple problem. So, how the heck do I handle this? What is the proper way of starting a task, forgetting about it, but guaranteeing it runs to completion?

Edit 1, in case anyone suggests it: I've read posts suggesting that _ = Task.Run(async () => await someFunctionAsync()); may serve my needs, but this is not the case either. Though another thread runs the method, its context is lost as well and the task is cancelled, cancelling the child task.

Edit 2: I realize that the controller example is not necessarily the best, as I could simply write the code differently to respond immediately, then wait for the method to complete before disposing of the controller:

        [HttpGet]
        public async Task Get()
        {
            base.Response.StatusCode = 204;
            await base.Response.CompleteAsync(); //Returns 204 to caller here.
            await DoSomethingAsync();
        }

Answer 1

how can I guarantee that the task completes even if the context of the task is finalized/destroyed?

...

the tasks are cancelled when the parent controller is destroyed

...

Though another thread runs the method, its context is lost as well and the task is cancelled, cancelling the child task.

Your core question is about how to run a task that continues running after the request completes. So there is no way to preserve the request context. Any solution you use must copy any necessary information out of the request context before the request completes.

Plus, this seems convoluted and feels like a bad solution to a simple problem. So, how the heck do I handle this? What is the proper way of starting a task, forgetting about it, but guaranteeing it runs to completion?

That last part is the stickler: "guaranteeing it runs to completion". Discarding tasks, using Task.Run, and using an in-memory collection of in-progress tasks are all incorrect solutions in this case.

The only correct solution is even more convoluted than these relatively simple approaches: you need a basic distributed architecture (explained in detail on my blog). Specifically:

1. A durable queue (e.g., Azure Queue). This holds the serialized representation of the work to be done - including any values from the request context.
2. A background processor (e.g., Azure Function). I prefer independent background processors, but it's also possible to use BackgroundService for this.

The durable queue is the key; it's the only way to guarantee the tasks will be executed.

Answer 2

There's a lot to unpack here. I'll probably miss a few details, but let me share a few things that should set up a pretty good foundation.

Fundamentally, what it sounds like you're asking about is how to create background tasks in ASP.NET. In .NET 4.x, there was a QueueBackgroundWorkItem method created for this purpose: it gave your task a new cancellation token to use instead of the one provided by the controller action, and it switched you to a different context for the action you provided.

In asp.net core, there are more powerful (but more complicated) IHostedService implementations, including the BackgroundService, but there's nothing quite as simple as QueueBackgroundWorkItem. However, the docs include an example showing how you can use a BackgroundService to basically write your own implementation of the same thing. If you use their code, you should be able to inject an IBackgroundTaskQueue into your controller and call QueueBackgroundWorkItemAsync to enqueue a background task.

Both of these approaches take care of the need to have something await the tasks that get started. You can never truly "guarantee" completion of any given tasks, but they can at least handle the common use cases more gracefully. For example, they let your hosting environment (e.g. IIS) know that something is still running, so it doesn't automatically shut down just because no requests are coming in. And if the hosting environment is being instructed to shut down, it can signal that fact through the cancellation tokens and you can hopefully quickly get your task into a safe state for shutting down rather than being unceremoniously aborted.

They also handle the problem of uncaught exceptions in the background tasks: the exceptions are caught and logged instead of either being silently eaten or completely killing the application.

Neither of these do anything to help you maintain context about things like the current request or user. This is sensible, because the whole point is to allow an action to extend beyond the scope of any given request. So you'll need to write any code you call in these actions to not rely on HttpContext/IHttpContextAccessor or anything stateful like that. Instead, gather what information you need from the context prior to enqueueing the background task, and pass that information along as variables and parameters to downstream code. This is usually good practice anyway, since the HTTP Context is a responsibility that should stay in controller-level code, while most of your business logic should think in terms of business-level models instead. And relying on State is usually best avoided where possible, to create software that's more reliable, testable, etc.

For other types of applications, there are other approaches you'd need to take. Usually it's best to do an internet search for [framework] background tasks where [framework] is the framework you're working in (WPF, e.g.). Different frameworks will have different restrictions. For example, if you write a console app that expects to run without any interaction beyond the command-line arguments, the Task returned from your Main function will probably need to await all the tasks that you start therein. A WPF app, on the other hand, might kick off several background tasks when events like button clicks are invoked, but there are tricks to make sure you do CPU-intensive work on background threads while only interacting with UI elements while on the UI thread.