EF Core 2.1 to EF Core 8/.NET 8 migration: Why does this EF Core InMemory query generate a 'Nullable object must have a value' error

Question

After upgrading my .NET 5.0 project to .NET 8.0, the LINQ query now generates an exception

System.InvalidOperationException: Nullable object must have a value

The code below executes successfully in .NET 5.0 (with EF 2.1.1) but fails after upgrading to .NET 8.0 (with EF Core 8.0.12).

My DbContext and model are configured as shown here:

namespace Example
{
    public class Competitor
    {
        public string ID { get; set; }
    }

    public class Result
    {
        public string ID { get; set; }
        public string WinnerID { get; set; }
    }

    public class MyDbContext : DbContext
    {
        public MyDbContext()
            : base(new DbContextOptionsBuilder<MyDbContext>().UseInMemoryDatabase("MyDB").Options)
        {
        }

        public DbSet<Competitor> Competitors { get; set; }
        public DbSet<Result> Results { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<Competitor>()
                .HasKey(c => c.ID);

            modelBuilder.Entity<Result>()
                .HasKey(r => r.ID);
        }
    }
}

And here is the query that fails:

namespace Example
{
    public class UnitTest1
    {
        [Fact]
        public void TestOuterJoinToGroup()
        {
            using var context = new MyDbContext();

            context.Competitors.AddRange(new Competitor() { ID = "1" }, new Competitor() { ID = "2" });
            context.Results.AddRange(new Result() { ID = "1", WinnerID = "1" }, new Result() { ID = "2", WinnerID = "1" });
            context.SaveChanges();

            var individualSummary = context.Results.GroupBy(
                ic => ic.WinnerID,
                (WinnerID, ic) => new
                {
                    CompetitorID = WinnerID,
                    Wins = ic.Count()
                });

            var competitorResults = from c in context.Competitors
                                    join r in individualSummary on c.ID equals r.CompetitorID into groupJoin
                                    from outerJoin in groupJoin.DefaultIfEmpty()
                                    select new
                                    {
                                        Id = c.ID,
                                        Wins = outerJoin == null ? 0 : outerJoin.Wins,
                                    };
            var arrCompetitorResults = competitorResults.ToList();
        }
    }
}

What is the cause of the exception and what do I need to change to make the query execute successfully?

The full exception is:

System.InvalidOperationException: Nullable object must have a value.
   at lambda_method148(Closure, ValueBuffer)
   at System.Linq.Utilities.<>c__DisplayClass2_0`3.<CombineSelectors>b__0(TSource x)
   at System.Linq.Utilities.<>c__DisplayClass2_0`3.<CombineSelectors>b__0(TSource x)
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNextHelper()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Example.UnitTest1.TestOuterJoinToGroup() in D:\source\TestProject1\TestProject1\UnitTest1.cs:line 73

Answer 1

This is because until EF-core 2, EF auto switched to client-side evaluation when it encountered parts in a LINQ query that couldn't be translated into SQL. In your case, EFC 2.1.1, this apparently lead to client-side evaluation of the part outerJoin == null ? 0 : outerJoin.Wins. I'm not really sure, because in EFC 2.2.6 (the earliest version I can test easily) I get the same exception.

As of EF core 3, EF only supports client-side evaluation in the final Select but has also become smarter in evaluating the LINQ query before translation. Now it concludes that outerJoin.Wins is an int (not nullable) and it concludes that, technically, the outerJoin == null condition is always false and doesn't need translation. However, the data still gives rise to results where outerJoin.Wins is null.

This is an infamous glitch that can be hard to spot in more complex queries, but the solution is always to help EF by telling it that the result is nullable. In your query, this is done by the following construct:

var competitorResults = from c in context.Competitors
                        join r in individualSummary on c.ID equals r.CompetitorID into groupJoin
                        from outerJoin in groupJoin.DefaultIfEmpty()
                        select new
                        {
                            Id = c.ID,
                            Wins = (int?)outerJoin.Wins ?? 0,
                        };

Here, (int?)outerJoin.Wins tells EF that outerJoin.Wins can be null and ?? 0 tells it to convert null values to 0. In the SQL query this appears as COALESCE([p0].[Wins], 0) AS [Wins].