AVCodec h264_v4l2m2m hardware decoding unable to find device

Question

Using a custom compiled FFmpeg:

 $ ./ffmpeg -codecs | grep h264
ffmpeg version n6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
  configuration: --arch=aarch64 --enable-cross-compile --target-os=linux --cross-prefix=aarch64-linux-gnu- --prefix=/builds/dronesense/rust/ffmpeg-build/ffmpeg/out --pkgconfigdir= --pkg-config=pkg-config --extra-libs='-ldl -lpthread' --enable-libvpx --enable-libx264 --enable-libx265 --enable-decklink --enable-gpl --enable-nonfree --enable-shared --disable-static
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
 DEV.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_v4l2m2m )

/dev/video32 seems to have H.264 decoding support:

$ v4l2-ctl --list-formats-out -d /dev/video32
ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Output Multiplanar
    Pixel Format: 'MPG2' (compressed)
    Name        : MPEG-2 ES

    Index       : 1
    Type        : Video Output Multiplanar
    Pixel Format: 'H264' (compressed)
    Name        : H.264

    Index       : 2
    Type        : Video Output Multiplanar
    Pixel Format: 'HEVC' (compressed)
    Name        : HEVC

    Index       : 3
    Type        : Video Output Multiplanar
    Pixel Format: 'VP80' (compressed)
    Name        : VP8

    Index       : 4
    Type        : Video Output Multiplanar
    Pixel Format: 'VP90' (compressed)
    Name        : VP9

I've tried two approaches (Rust with bindgen):

Approach 1:

fn init_decoder() -> Arc<ContextWrapper> {
    let name = CString::new("h264_v4l2m2m").unwrap();
    let codec = unsafe { ffmpeg::avcodec_find_decoder_by_name(name.as_ptr()) };
    if codec.is_null() {
        error!("finding codec");
        process::exit(1);
    }

    let ctx = unsafe { ffmpeg::avcodec_alloc_context3(codec) };
    if ctx.is_null() {
        error!("creating context");
        process::exit(1);
    }

    let r = unsafe { ffmpeg::avcodec_open2(ctx, codec, ptr::null_mut()) };
    if r < 0 {
        error!("opening codec: {r}");
        process::exit(1);
    }

    Arc::new(ContextWrapper(ctx))
}

Results in:

[h264_v4l2m2m @ 0x7f1c001600] Could not find a valid device
[h264_v4l2m2m @ 0x7f1c001600] can't configure decoder
[ERROR] [decoder] [webrtc::codec] opening codec: -1

Approach 2

fn init_decoder() -> Arc<ContextWrapper> {
    let name = CString::new("h264_v4l2m2m").unwrap();
    let codec = unsafe { ffmpeg::avcodec_find_decoder_by_name(name.as_ptr()) };
    if codec.is_null() {
        error!("finding codec");
        process::exit(1);
    }

    let mut i = 0;
    let mut hw_pix_fmt: AVPixelFormat = unsafe { mem::zeroed() };
    loop {
        let config = unsafe { ffmpeg::avcodec_get_hw_config(codec, i) };
        if config.is_null() {
            error!("decoder not supported");
            process::exit(1);
        }

        unsafe {
            info!("device type: {:?}", (*config).device_type);
            if ((*config).methods & ffmpeg::AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX as i32) > 0 {
                hw_pix_fmt = (*config).pix_fmt;
                break;
            }
        }
    }

    info!("pixel format: {:?}", hw_pix_fmt);

    let ctx = unsafe { ffmpeg::avcodec_alloc_context3(codec) };
    if ctx.is_null() {
        error!("creating context");
        process::exit(1);
    }

    let r = unsafe { ffmpeg::avcodec_open2(ctx, codec, ptr::null_mut()) };
    if r < 0 {
        error!("opening codec: {r}");
        process::exit(1);
    }

    Arc::new(ContextWrapper(ctx))
}

Results in: error!("decoder not supported");

I feel like there is a major step missing because looking at FFmpeg's Hardware Decode Example there are looking for a device type, to which v4l2 is not a part of the enum, so I do not what functions to call to get it setup.

What is the proper way to setup an AVCodec decoder with H.264 hardware acceleration for v4l2m2m?