ASP.Net MVC async with LINQ Lambda select results in "A second operation was started on this context before a previous operation completed."

Question

I have been battling with what is proving to be a diffuclt to track down issue involving awaited tasks within a LINQ Lambda select.

Note: The code below has been simplified and uses demo naming i.e. Town, Region etc. But the workflow is the same as the original code in which the issue occurs.

• Code runs perfectly when stepping in VS debugger.
• But when running in non-debug mode if a town is > level 1 and the await this.CheckTownSpecialProperty(t) function is called, I end up getting A second operation was started on this context before a previous operation completed..
• This error is only thrown when running in non-debug mode. In debug mode it works fine, presumably because stepping prevents the concurrency clash on the context.
• I have read the Microsoft warnings about the dangers of async within LINQ Lambda selects caused by the deferred execution. https://learn.microsoft.com/en-us/dotnet/csharp/async
• I basically just want to know why it happens in this case even with the usage of the using blocks controlling the contexts.

        public async Task<IEnumerable<TownSearchOutputDto>> GetTowns(TownSearchInputDto input)
        {
            var numResults = 25;

            var query = _townRepository.GetAll()
                                          .Include(t => t.Tests)
                                          .Include(t => t.Region.Select(s => s.SubRegion))
                                          .Include(t => t.Province); // Eager load relations.

            // Filter based on CountyId.
            if (input.CountyId > 0)
            {
                query = query.Where(t => t.CountyId == input.CountyId);
            }

            var townList = await query
                                    .OrderBy(t => t.Name)
                                    .Take(numResults)
                                    .ToListAsync();

            IEnumerable<Task<TownSearchOutputDto>> tasks = townList.Select(async t => {
                var townLevel = this.GetTownLevel(t);
                bool hasSomeSpecialTownProperty = false;
                if (townLevel.Key > 1)
                {
                    hasSomeSpecialTownProperty = await this.CheckTownSpecialProperty(t);
                }
                return new TownSearchOutputDto
                {
                    Id = t.Id,
                    CountyId = t.CountyId,
                    RegionName = t.Region.Name,
                    SubRegionName = t.SubRegion.Name,
                    ProvinceName = t.Province.Name,
                    TownLevelOrder = townLevel.Key,
                    TownLevelName = townLevel.Value,
                    HasSomeSpecialTownProperty = hasSomeSpecialTownProperty,
                    Disabled = townLevel.Key > 2 || t.Tests.Any(a => a.TownId == t.Id)
                };
            }).ToList();

            var towns = await Task.WhenAll(tasks);

            return towns;
        }

        private KeyValuePair<int, string> GetTownLevel(Town town)
        {
            if (!string.IsNullOrWhiteSpace(town.Tiny))
            {
                return new KeyValuePair<int, string>(3, "Tiny");
            }
            else if (!string.IsNullOrWhiteSpace(town.Small))
            {
                return new KeyValuePair<int, string>(2, "Small");
            }
            else if (!string.IsNullOrWhiteSpace(town.Big))
            {
                return new KeyValuePair<int, string>(1, "Big");
            }
            return new KeyValuePair<int, string>(-1, null);
        }

        private async Task<bool> CheckTownSpecialProperty(Town town)
        {
            // If no town is found then we cannot continue.
            if (town is null)
            {
                throw new UserFriendlyException("Town not found. Cannot continue!");
            }

            var townLevel = this.GetTownLevel(town);

            if(townLevel.Key == 1) // This is big town level, exit early. We are just testing against smaller towns.
            {
                return false;
            }

            // Find any big town of the same name also being tested.
            // This is where the issue occurs.
            // In debug mode all works well but when running freely, I get: 
            // "A second operation was started on this context before a previous operation completed."
            var townList = await _townRepository.GetAll()
                                            .Include(t => t.Tests)
                                            .Include(t => t.Region.Select(s => s.SubRegion))
                                            .Include(t => t.Province); // Eager load relations.
                .Where(t => t.Id != town.Id
                                && t.Region.Name == town.Region.Name
                                && t.Name == town.Name
                                && t.SubRegion.Name == town.SubRegion.Name
                                && t.Tests.Any(a => a.TownId == t.Id))
                .ToListAsync();

            // If we have found a big town also being tested return true.
            if (townList.Any(t => this.GetTownLevel(t).Key == 1))
            {
                return true;
            }

            return false;
        }

Things I have tried to resolve the issue

1. Making CheckTownSpecialProperty() synchronous. But this defeats the purpose of GetTowns being asynchronous to begin with, it was really just to let me know the funtion was working OK.
2. I tried to add using blocks and control the context instances manually. I surmised that this would work since the context instances would be recreated for each task.
   • But this results in: "The operation cannot be completed because the DbContext has been disposed."
   • I also tried the same in the GetTowns() function and the same "The operation cannot be completed because the DbContext has been disposed." error is thrown.

        List<Town> townList = null;
        using (var context = _abpContext.GetDbContext())
            {     
                townList = await context.Towns
                                .Include(t => t.Tests)
                                .Include(t => t.Region.Select(s => s.SubRegion))
                                .Include(t => t.Province); // Eager load relations.
                .Where(t => t.Id != town.Id
                                && t.Region.Name == town.Region.Name
                                && t.Name == town.Name
                                && t.SubRegion.Name == town.SubRegion.Name
                                && t.Tests.Any(a => a.TownId == t.Id))
                .ToListAsync();
            }

3. Turning off Lazy Loading on the contexts in the using block via context.Configuration.LazyLoadingEnabled = false;. I was hoping that fully materialising the objects would prevent the error, but this made no difference.
4. Adding AsNoTracking() to the queries. This was a last try to see if I could fully materialise the objects and prevent the concurrency issues. Sadly it did not work.

Answer 1

Your problem is in GetTowns. Here's what that code is doing:

1. Creates a query.
2. Asynchronously executes that query against _townRepository to get townList.
3. Spins up several tasks, where several of them may be calling CheckTownSpecialProperty.
4. Asynchronously waits for all those tasks to complete using Task.WhenAll.

So CheckTownSpecialProperty may be called concurrently, and all those concurrent executions use the same _townRepository. This is not allowed.

To fix this, either:

1. Make your original query more complex (e.g., doing a join) so that a second lookup isn't necessary.
2. New up a repository within CheckTownSpecialProperty, so that each of the concurrent executions have their own repository.
3. Change the logic so that the secondary queries in CheckTownSpecialProperty are run one at a time. They can still be asynchronous, but serial.
4. Combine the secondary queries into a single secondary query.