CODEWITHSUNDEEP
entity-frameworkasp.net-core.net-coreentity-framework-coreef-core-3.0
user9807202
user9807202

Reputation: 253

.NET Core / EF Core (SQL) Max Pool issue when using (Hosted) Background Service

My team and I have been experiencing some EF Core / SQL pooling issues on a high utilization .NET Core Web App.

Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.

The issue started occurring when EF Core / ApplicationDbContext was being referenced in a Background task on a hosted service (QueuedHostedService). Following the guide at:

https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-3.1&tabs=visual-studio

I've followed the recommended dependency injection and hosted service guides.

Steps to reproduce

The relevant lines:

Startup.cs 

...
services.AddDbContext<ApplicationDbContext>(options =>
                options.UseSqlServer(Configuration.GetConnectionString("DefaultConnection")));

...

services.AddScoped<IScopedWorker, ScopedWorker>();
services.AddSingleton<MySingletonThatAddsToBackground>();
// Follow the host service guide from microsoft.
services.AddHostedService<QueuedHostedService>();
services.AddSingleton<IBackgroundTaskQueue, BackgroundTaskQueue>();
MySingletonThatAddsToBackground.cs 
public class MySingletonThatAddsToBackground
{
    private readonly IBackgroundTaskQueue _taskQueue;
    private readonly ILogger _logger;
    private readonly CancellationToken _cancellationToken;
    public IServiceProvider _services { get; }

    public MySingletonThatAddsToBackground(IServiceProvider services, IBackgroundTaskQueue taskQueue,
        ILogger<MonitorLoop> logger,
        IHostApplicationLifetime applicationLifetime)
    {
        _services = services;
        _taskQueue = taskQueue;
        _logger = logger;
        _cancellationToken = applicationLifetime.ApplicationStopping;
    }

    public void DoWorkBackground()
    {
        // Enqueue a background work item
        _taskQueue.QueueBackgroundWorkItem(async token =>
        {
            try
            {
                using (var scope = _services.CreateScope())
                {
                    var scopedWorker = scope.ServiceProvider.GetRequiredService<IScopedWorker>();

                    await scopedWorker.DoWork();
                }
            }
            catch (OperationCanceledException)
            {
                // Prevent throwing if the Delay is cancelled
            }
        });
    }
}
ScopedWorker.cs 
public class ScopedWorker : IScopedWorker
{
    private readonly ApplicationDbContext _db;

    public ScopedWorker(ApplicationDbContext db)
    {
        _db = db;
    }

    public void DoWork()
    {
        var customers = _db.MyCustomers.ToListAsync();

        // Do stuff to customers.

        await _db.SaveChangesAsync();
    }
}

Discussion

Inside MySingletonThatAddsToBackground, after the using (var scope) is completed, shouldn't the scope be disposed, which then disposes ScopedWorker (scoped), which then disposes the ApplicationDbConext (scoped), and then closes the connection / pool connection?

Is there something I'm not implementing properly to cause the connection pool leak?

Further technical details

EF Core version: 3.1.4 Database provider: Microsoft.EntityFrameworkCore.SqlServer Target framework: .NET Core 3.1.4 Operating system: Windows Server 2016, SQL Server 2016 IDE: 16.6

Upvotes: 5

Views: 2910

Answers (1)

Antonio Leonardo
Antonio Leonardo

Reputation: 1862

I read this article recently (at ASP.NET Monsters) and I think this can help for your case: Please, try to monitoring the Background Service using NETSTAT and look the State of each request; in article contains more explained detailing, but, in resuming, problems associate a erroneous behavior at Garbage Dispose routine, that impacts the ends of HTTP Pipeline establishment.

In other words, check (by NESTAT) if State is different of ESTABLISHED; if it's occurs, you'll get the root cause and will need to change your code approach, being careful about using garbage collection.

Upvotes: -1

Related Questions