Why Enum's HasFlag method need boxing?

Question

I am reading "C# via CLR" and on page 380, there's a note saying the following:

Note The Enum class defines a HasFlag method defined as follows

public Boolean HasFlag(Enum flag);

Using this method, you could rewrite the call to Console.WriteLine like this:

Console.WriteLine("Is {0} hidden? {1}", file, attributes.HasFlag(FileAttributes.Hidden));

However, I recommend that you avoid the HasFlag method for this reason:

Since it takes a parameter of type Enum, any value you pass to it must be boxed, requiring a memory allocation ."

I can not understand this bolded statement -- why "

any value you pass to it must be boxed

The flag parameter type is Enum, which is a value type, why would there be boxing? The "any value you pass to it must be boxed" should mean boxing happens when you pass value type to parameter Enum flag, right?

Answer 1

The type is the abstract base class of all enum types (this is distinct and different from the underlying type of the enum type), and the members inherited from are available in any enum type. A boxing conversion (§10.2.9) exists from any enum type to , and an unboxing conversion (§10.3.7) exists from to any enum type.

Note that is not itself an enum_type. Rather, it is a class_type from which all enum_types are derived. The type inherits from the type (§8.3.2), which, in turn, inherits from type . At run-time, a value of type can be or a reference to a boxed value of any enum type.

https://learn.microsoft.com/zh-cn/dotnet/csharp/language-reference/language-specification/enums#the-systemenum-type

Answer 2

As suggested by Timo the solution of Martin Tilo Schmitz can be implemented without the need for the /unsafe switch:

public static bool HasAnyFlag<E>(this E lhs, E rhs) where E : unmanaged, Enum
{
    switch (Unsafe.SizeOf<E>())
    {
    case 1:
        return (Unsafe.As<E, byte>(ref lhs) & Unsafe.As<E, byte>(ref rhs)) != 0;
    case 2:
        return (Unsafe.As<E, ushort>(ref lhs) & Unsafe.As<E, ushort>(ref rhs)) != 0;
    case 4:
        return (Unsafe.As<E, uint>(ref lhs) & Unsafe.As<E, uint>(ref rhs)) != 0;
    case 8:
        return (Unsafe.As<E, ulong>(ref lhs) & Unsafe.As<E, ulong>(ref rhs)) != 0;
    default:
        throw new Exception("Size does not match a known Enum backing type.");
    }
}

The NuGet System.Runtime.CompilerServices.Unsafe is required to compile this with .NET Framework.

Performance

• I have to mention that any generic implementation is still almost an order of magnitude slower than a native check, i.e. ((int)lhs & (int)rhs) != 0. I guess taking the reference of lhs, rhs prevents optimization of the storage of the function variables. The runtime dispatch of the enum size adds another overhead.
• But it is still one order of magnitude faster than HasFlag.
• And well, the performance benefit is almost zero comapred to HasFlag if optimizations are turned off in a debug build.
• There is no significant difference between using unsafe { } and using class Unsafe in optimized (release) builds. Only without optimizations class Unsafe is almost as slow as HasFlag.
• Using a delegate as supercat recommended is no reasonable option since the resulting function pointer call is slow on most CPU architectures and even more it completely prevents inlining.
• MethodImplOptions.AggressiveInlining adds no value.

Conclusion

There is still no really fast and readable implementation for testing flags in enums.

Answer 3

Since C# 7.3, where generic Enum constraint was introduced, you can write a fast, non allocating version that doesn't rely on reflection. It requires the compiler flag /unsafe but since Enum backing types can only be a fixed amount of sizes, it should be perfectly safe to do:

using System;
using System.Runtime.CompilerServices;
public static class EnumFlagExtensions
{
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static bool HasFlagUnsafe<TEnum>(TEnum lhs, TEnum rhs) where TEnum : unmanaged, Enum
    {
        unsafe
        {
            switch (sizeof(TEnum))
            {
                case 1:
                    return (*(byte*)(&lhs) & *(byte*)(&rhs)) > 0;
                case 2:
                    return (*(ushort*)(&lhs) & *(ushort*)(&rhs)) > 0;
                case 4:
                    return (*(uint*)(&lhs) & *(uint*)(&rhs)) > 0;
                case 8:
                    return (*(ulong*)(&lhs) & *(ulong*)(&rhs)) > 0;
                default:
                    throw new Exception("Size does not match a known Enum backing type.");
            }
        }
    }
}

Answer 4

It's worth noting that a generic HasFlag<T>(T thing, T flags) which is about 30 times faster than the Enum.HasFlag extension method can be written in about 30 lines of code. It can even be made into an extension method. Unfortunately, it's not possible in C# to restrict such a method to only take things of enumerated types; consequently, Intellisense will pop up the method even for types for which it is not applicable. I think if one used some language other than C# or vb.net to write the extension method it might be possible to make it pop up only when it should, but I'm not familiar enough with other languages to try such a thing.

internal static class EnumHelper<T1>
{
    public static Func<T1, T1, bool> TestOverlapProc = initProc;
    public static bool Overlaps(SByte p1, SByte p2) { return (p1 & p2) != 0; }
    public static bool Overlaps(Byte p1, Byte p2) { return (p1 & p2) != 0; }
    public static bool Overlaps(Int16 p1, Int16 p2) { return (p1 & p2) != 0; }
    public static bool Overlaps(UInt16 p1, UInt16 p2) { return (p1 & p2) != 0; }
    public static bool Overlaps(Int32 p1, Int32 p2) { return (p1 & p2) != 0; }
    public static bool Overlaps(UInt32 p1, UInt32 p2) { return (p1 & p2) != 0; }
    public static bool Overlaps(Int64 p1, Int64 p2) { return (p1 & p2) != 0; }
    public static bool Overlaps(UInt64 p1, UInt64 p2) { return (p1 & p2) != 0; }
    public static bool initProc(T1 p1, T1 p2)
    {
        Type typ1 = typeof(T1);
        if (typ1.IsEnum) typ1 = Enum.GetUnderlyingType(typ1);
        Type[] types = { typ1, typ1 };
        var method = typeof(EnumHelper<T1>).GetMethod("Overlaps", types);
        if (method == null) method = typeof(T1).GetMethod("Overlaps", types);
        if (method == null) throw new MissingMethodException("Unknown type of enum");
        TestOverlapProc = (Func<T1, T1, bool>)Delegate.CreateDelegate(typeof(Func<T1, T1, bool>), method);
        return TestOverlapProc(p1, p2);
    }
}
static class EnumHelper
{
    public static bool Overlaps<T>(this T p1, T p2) where T : struct
    {
        return EnumHelper<T>.TestOverlapProc(p1, p2);
    }
}

EDIT: A previous version was broken, because it used (or at least tried to use) EnumHelper<T1, T1>.

Answer 5

Moreover, there's more than single boxing in Enum.HasFlag:

public bool HasFlag(Enum flag)
{
    if (!base.GetType().IsEquivalentTo(flag.GetType()))
    {
        throw new ArgumentException(Environment.GetResourceString("Argument_EnumTypeDoesNotMatch", new object[]
        {
            flag.GetType(),
            base.GetType()
        }));
    }
    ulong num = Enum.ToUInt64(flag.GetValue());
    ulong num2 = Enum.ToUInt64(this.GetValue());
    return (num2 & num) == num;
}

Look at GetValue method calls.

Update. Looks like MS had optimized this method in .NET 4.5 (the source code has been downloaded from referencesource):

    [System.Security.SecuritySafeCritical]
    public Boolean HasFlag(Enum flag) { 
        if (flag == null)
            throw new ArgumentNullException("flag"); 
        Contract.EndContractBlock(); 

        if (!this.GetType().IsEquivalentTo(flag.GetType())) { 
            throw new ArgumentException(Environment.GetResourceString("Argument_EnumTypeDoesNotMatch", flag.GetType(), this.GetType()));
        }

        return InternalHasFlag(flag); 
    }

    [System.Security.SecurityCritical]  // auto-generated 
    [ResourceExposure(ResourceScope.None)]
    [MethodImplAttribute(MethodImplOptions.InternalCall)] 
    private extern bool InternalHasFlag(Enum flags);

Answer 6

_{In this instance, two boxing calls are required before you even get into the HasFlags method. One is for resolving the method call on the value type to the base type method, the other is passing the value type as a reference type parameter. You can see the same in IL if you do var type = 1.GetType();, the literal int 1 is boxed before the GetType() call. The boxing on method call seems to be only when methods are not overridden in the value type definition itself, more can be read here: Does calling a method on a value type result in boxing in .NET?}

The HasFlags takes an Enum class argument, so the boxing will occur here. You are trying to pass what is a value type into something expecting a reference type. To represent values as references, boxing occurs.

There is lots of compiler support for value types and their inheritance (with Enum / ValueType) that confuses the situation when trying to explain it. People seem to think that because Enum and ValueType is in the inheritance chain of value types boxing suddenly doesn't apply. If this were true, the same could be said of object as everything inherits that - but as we know this is false.

This doesn't stop the fact that representing a value type as a reference type will incur boxing.

And we can prove this in IL (look for the box codes):

class Program
{
    static void Main(string[] args)
    {
        var f = Fruit.Apple;
        var result = f.HasFlag(Fruit.Apple);

        Console.ReadLine();
    }
}

[Flags]
enum Fruit
{
    Apple
}



.method private hidebysig static 
    void Main (
        string[] args
    ) cil managed 
{
    // Method begins at RVA 0x2050
    // Code size 28 (0x1c)
    .maxstack 2
    .entrypoint
    .locals init (
        [0] valuetype ConsoleApplication1.Fruit f,
        [1] bool result
    )

    IL_0000: nop
    IL_0001: ldc.i4.0
    IL_0002: stloc.0
    IL_0003: ldloc.0
    IL_0004: box ConsoleApplication1.Fruit
    IL_0009: ldc.i4.0
    IL_000a: box ConsoleApplication1.Fruit
    IL_000f: call instance bool [mscorlib]System.Enum::HasFlag(class [mscorlib]System.Enum)
    IL_0014: stloc.1
    IL_0015: call string [mscorlib]System.Console::ReadLine()
    IL_001a: pop
    IL_001b: ret
} // end of method Program::Main

The same can be seen when representing a value type as ValueType, it also results in boxing:

class Program
{
    static void Main(string[] args)
    {
        int i = 1;
        ValueType v = i;

        Console.ReadLine();
    }
}


.method private hidebysig static 
    void Main (
        string[] args
    ) cil managed 
{
    // Method begins at RVA 0x2050
    // Code size 17 (0x11)
    .maxstack 1
    .entrypoint
    .locals init (
        [0] int32 i,
        [1] class [mscorlib]System.ValueType v
    )

    IL_0000: nop
    IL_0001: ldc.i4.1
    IL_0002: stloc.0
    IL_0003: ldloc.0
    IL_0004: box [mscorlib]System.Int32
    IL_0009: stloc.1
    IL_000a: call string [mscorlib]System.Console::ReadLine()
    IL_000f: pop
    IL_0010: ret
} // end of method Program::Main

Answer 7

There are two boxing operations involved in this call, not just one. And both are required for one simple reason: Enum.HasFlag() needs type information, not just values, for both this and flag.

Most of the time, an enum value truly is just a set of bits and the compiler has all the type information it needs from the enum types represented in the method signature.

However, in the case of Enum.HasFlags() the very first thing it does is call this.GetType() and flag.GetType() and make sure they're identical. If you wanted the typeless version, you'd be asking if ((attribute & flag) != 0), instead of calling Enum.HasFlags().

Answer 8

Enum inherits from ValueType which is... a class! Hence the boxing.

Note that the Enum class can represents any enumeration, whatever its underlying type is, as a boxed value. Whereas a value such as FileAttributes.Hidden will be represented as real value type, int.

Edit: let's differentiate the type and the representation here. An int is represented in memory as 32 bits. Its type derives from ValueType. As soon as you assign an int to an object or derived class (ValueType class, Enum class), you're boxing it, effectively changing its representation to a class now containing that 32 bits, plus additional class information.

Answer 9

When ever you pass a value type of a method that takes object as a parameter, as in the case of console.writeline, there will be an inherent boxing operation. Jeffery Richter discusses this in detail in the same book you mention.

In this case you are using the string.format method of console.writeline, and that takes a params array of object[]. So your bool, will be cast to object, so hence you get a boxing operation. You can avoid this by calling .ToString() on the bool.