CODEWITHSUNDEEP
rust
ajp
ajp

Reputation: 2463

Is mem::transmute between enum and tagged union representation safe?

The official rust documentation guarantees the layout of a repr(C, u8) enum (with fields), in terms of an equivalent set of types implementing a manually tagged union. If I define both implementations, am I justified in interconverting between the two with mem::transmute?

The motivation is that in most of my code I want to use a nice ergonomic rust enum, but in one or two places I need to do unsafe operations on the tagged union representation.

/// BEGIN copy-paste from
/// https://doc.rust-lang.org/reference/type-layout.html#combining-primitive-representations-of-enums-with-fields-and-reprc

// This Enum has the same representation as ...
#[repr(C, u8)]
enum MyEnum {
    A(u32),
    B(f32, u64),
    C { x: u32, y: u8 },
    D,
 }

// ... this struct.
#[repr(C)]
struct MyEnumRepr {
    tag: MyEnumDiscriminant,
    payload: MyEnumFields,
}

// This is the discriminant enum.
#[repr(u8)]
enum MyEnumDiscriminant { A, B, C, D }

// This is the variant union.
#[repr(C)]
union MyEnumFields {
    A: MyAFields,
    B: MyBFields,
    C: MyCFields,
    D: MyDFields,
}

#[repr(C)]
#[derive(Copy, Clone)]
struct MyAFields(u32);

#[repr(C)]
#[derive(Copy, Clone)]
struct MyBFields(f32, u64);

#[repr(C)]
#[derive(Copy, Clone)]
struct MyCFields { x: u32, y: u8 }

// This struct could be omitted (it is a zero-sized type), and it must be in
// C/C++ headers.
#[repr(C)]
#[derive(Copy, Clone)]
struct MyDFields;

/// END copy-paste

fn as_tagged_union(x: &mut MyEnum) -> &mut MyEnumRepr {
  unsafe { core::mem::transmute(x) }
}

fn as_enum(x: &mut MyEnumRepr) -> &mut MyEnum {
  unsafe { core::mem::transmute(x) }
}

Playground

Upvotes: 1

Views: 176

Answers (2)

isaactfa
isaactfa

Reputation: 6651

As per the RFC that implements this, yes it is defined:

For both layouts, it is defined for Rust programs to cast/reinterpret/transmute such an enum into the equivalent Repr definition. Separately manipulating the tag and payload is also defined. The tag and payload need only be in a consistent/initialized state when the value is matched on (which includes Dropping it).
For instance, this code is valid (using the same definitions above):

/// Tries to parse a `#[repr(C, u8)] MyEnum` from a custom binary format, overwriting `dest`.
/// On Err, `dest` may be partially overwritten (but will be in a memory-safe state)
fn parse_my_enum_from<'a>(dest: &'a mut MyEnum, input: &mut &[u8]) -> Result<(), &'static str> {
    unsafe {
        // Convert to raw repr
        let dest: &'a mut MyEnumRepr = mem::transmute(dest);

        // If MyEnum was non-trivial, we might match on the tag and 
        // drop_in_place the payload here to start.

        // Read the tag
        let tag = input.get(0).ok_or("Couldn't Read Tag")?;
        dest.tag = match tag {
            0 => MyEnumTag::A,
            1 => MyEnumTag::B,
            2 => MyEnumTag::C,
            3 => MyEnumTag::D,
            _ => { return Err("Invalid Tag Value"); }
        };
        *input = &input[1..];

        // Note: it would be very bad if we panicked past this point, or if
        // the following methods didn't initialize the payload on Err!

        // Read the payload
        match dest.tag {
            MyEnumTag::A => parse_my_enum_a_from(&mut dest.payload.A, input),
            MyEnumTag::B => parse_my_enum_b_from(&mut dest.payload.B, input),
            MyEnumTag::C => parse_my_enum_c_from(&mut dest.payload.C, input),
            MyEnumTag::D => { Ok(()) /* do nothing */ }
        }
    }
}

Upvotes: 3

kmdreko
kmdreko

Reputation: 60072

Yes, you can std::mem::transmute between types that have the same layout and representation. The example you show from the Rust Reference shows how a #[repr(C)] enum is structured and how to mimic it with separate tag and union.

Since you are transmuting references, you have to ensure the alignment matches as well as structure but if following the guidance above that should be the same. See the nomicon for more guidance.

Upvotes: 1

Related Questions