CODEWITHSUNDEEP
c#asp.net-web-apiqueue
appsecguy
appsecguy

Reputation: 1049

How to queue background tasks in ASP.NET Web API

I have a webapi that is designed to process reports in a queue fashion. The steps the application takes are as follows:

I was thinking to use Entity Framework to create a database of queued items, such as:

public class EFBatchItem
{
    [Key]
    public string BatchId { get; set; }
    public DateTime DateCreated { get; set; }
    public DateTime DateCompleted { get; set; }
    public string BatchItem { get; set; }
    public BatchStatus Status { get; set; }
}

My question - Is there a more efficient way, using NServiceBus, BlockingCollection or ConcurrentQeueue, than to constantly poll the database and pull out pending items one by one? I have not used queues before.

One thought is to create a queue of tasks, and on a separate thread process all pending tasks. Somewhat similar to Most efficient way to process a queue with threads but I want to ensure that I am going the most efficient route.

EDIT: A big question I have here is the best way to display the progress to the user. Once the user submits content, he gets taken to a new page and can view the status by the batch identifier. Is MSMQ necessary, or NServiceBus, in order to notify the user? This seems like a variation of the REquest/Acknowledge/Push paradigm?

Upvotes: 20

Views: 29394

Answers (4)

menxin
menxin

Reputation: 2244

If you use Asp.net Core,you can use IHostService.

Upvotes: 1

Anytoe
Anytoe

Reputation: 1675

I like @Todd's solution. Just for the sake of completeness, but there's another option not mentioned yet:

HostingEnvironment.QueueBackgroundWorkItem(cancellationToken =>
{
    // Some long-running job
});

Note:

"When ASP.NET has to recycle, it will notify the background work (by setting a CancellationToken) and will then wait up to 30 seconds for the work to complete. If the background work doesn’t complete in that time frame, the work will mysteriously disappear."

And avoid using service methods with an injected DbContext here as this won't work.

Sources: MariusSchulz and StephenCleary

Upvotes: 2

tugberk
tugberk

Reputation: 58434

IMHO, your ASP.NET Web API application shouldn't run those background tasks by itself. It should only be responsible to receive the request, stick it inside the queue (as you indicated) and return the response indicating the success or failure of the received message. You can use variety of messaging systems such as RabbitMQ for this approach.

As for the notification, you have a few options. You can have an endpoint which the client can check whether the processing is completed or not. Alternatively, you could provide a streaming API endpoint which your client can subscribe to. This way, the client doesn't have to poll the server; the server can notify the clients which are connected. ASP.NET Web API has a great way of doing this. The following blog post explains how:

You can also consider SignalR for this type of server to client notifications.

The reason why background tasks are hard for ASP.NET Web API application is that you're responsible to keep the AppDomain alive. This is a hassle especially when you are hosting under IIS. The following blog posts explains really good what I mean:

Upvotes: 28

Kind Contributor
Kind Contributor

Reputation: 18513

It's a NuGet package and it's called HangFire - https://github.com/HangfireIO/Hangfire. The tasks persist even beyond apppool recycling.

Upvotes: 7

Related Questions