How do you properly return `null` from a generic marked with [return:MaybeNull]?

Question

In Microsoft's nullability documentation, there appears to be conflicting information.

On this page, it says the following (important part in bold/italic):

Generic definitions and nullability

Correctly communicating the null state of generic types and generic methods requires special care. The extra care stems from the fact that a nullable value type and a nullable reference type are fundamentally different. An int? is a synonym for Nullable<int>, whereas string? is string with an attribute added by the compiler. The result is that the compiler can't generate correct code for T? without knowing if T is a class or a struct.

This fact doesn't mean you can't use a nullable type (either value type or reference type) as the type argument for a closed generic type. Both List<string?> and List<int?> are valid instantiations of List.

What it does mean is that you can't use T? in a generic class or method declaration without constraints. For example, Enumerable.FirstOrDefault<TSource>(IEnumerable<TSource>) won't be changed to return T?. You can overcome this limitation by adding either the struct or class constraint. With either of those constraints, the compiler knows how to generate code for both T and T?.

Ok, so if you want to use T? in a generic, you have to constrain it to either a struct or class. simple enough.

But Then in the following page, they say this (again, emphasis in bold/italic):

Specify post-conditions: MaybeNull and NotNull

Suppose you have a method with the following signature:

public Customer FindCustomer(string lastName, string firstName)

You've likely written a method like this to return null when the name sought wasn't found. The null clearly indicates that the record wasn't found. In this example, you'd likely change the return type from Customer to Customer?. Declaring the return value as a nullable reference type specifies the intent of this API clearly.

For reasons covered under Generic definitions and nullability that technique does not work with generic methods. You may have a generic method that follows a similar pattern:

public T Find<T>(IEnumerable<T> sequence, Func<T, bool> predicate)

You can't specify that the return value is T? [but the] method returns null when the sought item isn't found. Since you can't declare a T? return type, you add the MaybeNull annotation to the method return:

[return: MaybeNull]
public T Find<T>(IEnumerable<T> sequence, Func<T, bool> predicate)

The preceding code informs callers that the contract implies a non-nullable type, but the return value may actually be null. Use the MaybeNull attribute when your API should be a non-nullable type, typically a generic type parameter, but there may be instances where null would be returned.

However...

Even copying that code straight from the documentation and giving it a default implementation that simply returns null, it won't compile!

[return: MaybeNull]
public T Find<T>(IEnumerable<T> sequence, Func<T, bool> predicate)
    => null;

I tried the null-forgiving operator, null!, also mentioned in the first-linked page (under the section 'Initialize the property to null') but that didn't work. You can't use default either because that doesn't return null for value types like int which return zero instead as shown here:

[return: MaybeNull]
public static T Find<T>(IEnumerable<T> sequence, Func<T, bool> predicate)
    => default;

var seq = new []{ 1, 2, 3 };
bool MyPredicate(int value) => false;
var x = Find(seq, MyPredicate);
Console.WriteLine($"X is {x}");

Output:

X is 0

So what am I missing here? How do you successfully implement their example code without resorting to using T? which requires type-constraining it to either class or struct? And if you did have to do that, then what's the point of MaybeNull?

Answer 1

Ok, so unfortunately, thanks to the difference between a nullable reference type and a nullable value type--a limitation that something like Swift doesn't have--what I'm after is not supported by C#.

Instead, as mentioned in my other answer, because of this you shouldn't use the 'returning null means no match' pattern. Instead, you should use a try-based pattern that returns a boolean and utilizes an out parameter for the value of interest (if any). Inside, you still use default (so you don't have to constrain T) but you have that extra boolean telling you whether the default itself should be ignored or not. This way will properly work for both class and struct types, and more importantly, for types such as int where the default is not null, but rather zero, it will help you differentiate between a predicate indicating it matched zero (return = true) or there was no passing predicate (return = false) and you can ignore that zero.

The trick is in using the NotNullWhen attribute to tell callers that the out parameter will never be null when the return value of the function is true, so as long as you check the return value before accessing the out parameter, you don't also have to check for null, and code-flow analysis will not display 'possible null' warnings either.

Here's the refactor of the above function...

public static bool TryFind<T>(this IEnumerable<T> items, Func<T, bool> predicate, [NotNullWhen(true)] out T result){
    
    foreach(var item in items){
        if(predicate(item)){
            result = item;
            return true;
        }
    }

    result = default;
    return false;
} 

Time to go refactor some old code!

Answer 2

Ok, experimenting more, there are two solutions to this. One, write two versions of your generic, one constrained to structs and the other to classes. You will have to change their name however because a constraint is not enough to differentiate function overloads.

The other way you don't have to constrain it, but you instead have to return default, not null in your implementation. But that still doesn't mean it will return null. It will only return null if that's the default for the passed-in type, meaning if you want null back, you have to also pre-process your non-nullable types into a nullable variant first before calling the generic.

While this works, I would call such an approach 'code-smell' and would instead change it to a 'Try'-based pattern. You would still return default, but as an out parameter, then return a boolean as the return value telling you if a match was actually found, or in other words, whether you should ignore that out parameter.

That way, for the non-nullable types, like int, you would still know there was no match due to the extra 'information' of the returned boolean which would address cases where zero itself could be a match. (i.e. the out as zero with a return as false means no match whereas an out as zero with a return of true means you found a match, zero.)

But back to the code...

Here's the example.

[return:MaybeNull] // Suppress compiler warning that 'default' may be null
public static T Find<T>(this IEnumerable<T> items, Func<T, bool> predicate){

    foreach(var item in items)
        if(predicate(item))
            return item;

    return default;
}

Now consider the following sequence of ints:

var seq = new []{ 1, 2, 3, 4 };

When you run this code...

bool MyIntPredicate(int value)
    => value == 5;

var x = seq.Find(MyIntPredicate);

Console.WriteLine($"X is {x}");

it outputs the following:

X is 0

However, if you want a null to indicate not found, you have to make T a nullable type, but that also means you need a predicate that takes a nullable type.

bool MyNullableIntPredicate(int? value)
    => value == 5;

// Convert the int array to a nullable-int array via `OfType`, then use the nullable-int predicate.
var y = seq.OfType<int?>().FirstOrDefault(MyNullableIntPredicate);

Console.WriteLine($"Y is {y?.ToString() ?? "null"}");

Running the above, you now get This outputs the following:

X is null

If you add 5 to the array, in both cases you get this...

X is 5

Answer 3

MaybeNull attribute doesn't change return value it is only only warning (usually for people) that indicates "not found value be returned - not talking about null". Why is that? You can return null of T right? You can't. Structures doesn't have null value. And returning just default value of structures (for example int = 0) doesn't indicates that the default value is not "not found value". You can use T? But it may be cases that you can't or don't want to return T? (Nullable) Maybe because inheritence or whatever. So you can notify caller about MaybeNot and they can threat returned value with special case

Nice reading: https://endjin.com/blog/2020/07/dotnet-csharp-8-nullable-references-maybenull

---

-----------Edit----------- based on comments

Looks like only way how to avoid making two sepperate generic classes for reference types and structs is making generic class unconstrained T and methods that can return null mark as "T?"

 public static class NullableGeneric<T>
    {
        public static Dictionary<int,T> Values { get; set; } = new Dictionary<int,T>();
#nullable enable       
        public static T? TryFind(int id) 
        {
            var isFound = Values.TryGetValue(id, out T value);
            return isFound ? value : default; // default of T? is null even for structs;
        }
#nullable disable
    
    }

You can test this with

 NullableGeneric<int?>.Values.Add(1, 5);
 int? structType = NullableGeneric<int?>.TryFind(0); // null
 int? structType2 = NullableGeneric<int?>.TryFind(1); // 5

based of this blog

C# 9 update: The ? operator now works on unconstrained generic types. So, you should use T? instead of the [MayBeNull] attribute.

Another sulltion would be returning new custom class wich contains bool "isNotFound" property and value property. (you can use LanguageExt.Core wich contains Option class - similar to Nullable but it has way more features and allows both reference types and structs)

to be honest i find whole nullablity thing very tricky