How to properly work with nested enums in Rust?

Question

I am new to Rust, so it is entirely possible that I'm trying a completely wrong approach, so please suggest a better one if necessary :)

Say, I'm writing a CPU emulator, and I want to organize functions that implement assembly instructions in a nice and convenient way, by using enums to define instructions and their addressing mode variants.

My first thought was to implement instruction decoder as a function like pub fn scan_instr(mem: &[u8], pc: &mut usize) (where mem represents a memory block with binary code, and pc is an instruction pointer), and represent all opcodes as a nested enum, like so:

pub enum Opcode {
    ADC(ADC), // add with carry
    AND(AND), // and (with accumulator)
    ASL(ASL), // arithmetic shift left
    BCC(BCC), // branch on carry clear
    BCS(BCS), // branch on carry set
    // and so on...
}

// Instruction enum with addressing mode variants
pub enum ADC {
    IMM = 0x69,  // ADC #oper,    2 bytes
    ZP = 0x65,   // ADC oper,     2 bytes
    ZPX = 0x75,  // ADC oper,X,   2 bytes
    ABS = 0x6D,  // ADC oper,     3 bytes
    ABSX = 0x7D, // ADC oper,X,   3 bytes
    ABSY = 0x79, // ADC oper,Y,   3 bytes
    INDX = 0x61, // ADC (oper,X), 2 bytes
    INDY = 0x71  // ADC (oper),Y, 2 bytes
}

pub enum AND {
    IMM = 0x29,  // AND #oper,    2 bytes
    ZP = 0x25,   // AND oper,     2 bytes
    ZPX = 0x35,  // AND oper,X,   2 bytes
    ABS = 0x2D,  // AND oper,     3 bytes
    ABSX = 0x3D, // AND oper,X,   3 bytes
    ABSY = 0x39, // AND oper,Y,   3 bytes
    INDX = 0x21, // AND (oper,X), 2 bytes
    INDY = 0x31  // AND (oper),Y, 2 bytes
}

// and so on...

Then, I would fetch next instruction byte from the memory array, use it as an Opcode enum variant, and match it to determine which instruction it is, then pass it to the corresponding function and let it further determine the particular addressing mode by itself. While this would be trivial with a "flat" enum, it's not quite clear to me how to do it with such nested enum in Rust (if even possible). I suspect this might be a wrong approach entirely, any suggestions?

Answer 1

I would use cascading matches like this:

match opcode {
    Opcode::ADC(adc) => match adc {
        ADC::IMM  => adc_imm(),
        ADC::ZP   => adc_zp(),
        ADC::ZPX  => adc_zpx(),
        ADC::ABS  => adc_abs(),
        ADC::ABSX => adc_absx(),
        ADC::ABSY => adc_absy(),
        ADC::INDX => adc_indx(),
        ADC::INDY => adc_indy(),
    }
    Opcode::AND(and) => match and {
        AND::IMM  => and_imm(),
        AND::ZP   => and_zp(),
        AND::ZPX  => and_zpx(),
        AND::ABS  => and_abs(),
        AND::ABSX => and_absx(),
        AND::ABSY => and_absy(),
        AND::INDX => and_indx(),
        AND::INDY => and_indy(),
    }
    …
}

For the fastest assignment of the opcodes to their numerical values, an array [Opcode;256] would be suitable, where each Opcode is at the corresponding numeric index in the array.

Answer 2

Rust supports pattern matching so you could match the Opcode variant and pass the inner enum type to the function.

match opcode {
    Opcode::ADC(adc) => handle_adc(adc),
    Opcode::AND(and) => handle_and(and),
    _ => todo!(),
}

and the functions can match their enum

fn handle_adc(adc: ADC) {
    match adc {
        ADC::IMM => todo!(),
        ADC::ZP => todo!(),
        _ => todo!(),
    }
}

fn handle_and(and: AND) {
    match and {
        AND::IMM => todo!(),
        AND::ZP => todo!(),
        _ => todo!(),
    }
}

Playground

Edit

If your opcode starts as u8, you can implement converter using TryFrom trait.

pub struct OpcodeError;

impl TryFrom<(u8, u8)> for Opcode {
    type Error = OpcodeError;
    fn try_from(v: (u8, u8)) -> Result<Self, Self::Error> {
        match v.0 {
            // I just used random numbers since I don't know the real ones
            0x00 => Ok(Opcode::ADC(ADC::try_from(v.1)?)),
            0x11 => Ok(Opcode::AND(AND::try_from(v.1)?)),
            _ => Err(OpcodeError),
        }
    }
}
impl TryFrom<u8> for ADC {
    type Error = OpcodeError;
    fn try_from(v: u8) -> Result<Self, Self::Error> {
        match v {
            0x69 => Ok(ADC::IMM),
            0x65 => Ok(ADC::ZP),
            _ => Err(OpcodeError),
        }
    }
}

fn main() {
    let opcode = Opcode::try_from((0x00, 0x65)).unwrap();
    dbg!(&opcode);
}

You could use strum crate, specifically FromRepr trait to automatically derive these conversions. Although for the nested enum (Opcode), you'll have to manually implement conversion since the macro just fills the inner value with its default value. Also you might want to look at #[repr(u8)] macro if you need to assign value to your opcode variants.

Playground