CODEWITHSUNDEEP
rustfficompile-time-constantnull-terminated
DanielV
DanielV

Reputation: 663

include_str for null terminated string

I need to read a file into a null terminated string at compile time.

Working in Rust OpenGL. I have a shader source code stored in a separate file. The function that will eventually read the source is gl::ShaderSource from the gl crate. All it needs is a pointer to a null terminated string (the std::ffi::CStr type).

Typically the guides I have seen read the shader source file using include_str!, then at run time allocate a whole new buffer of length +1, then copy the original source into the new buffer and put the terminating 0 at the end. I'd like to avoid all that redundant allocating and copying and just have the correctly null terminated string at compile time.

I realize it is somewhat petty to want to avoid an extra allocation for a short shader file, but the principle could apply to many other types of larger constants.

While scrolling through suggested questions during the preview I saw this: How do I expose a compile time generated static C string through FFI?

which led me to this solution:

    let bytes1 = concat!(include_str!("triangle.vertex_shader"), "\0");
    let bytes2 = bytes1.as_bytes();
    let bytes3 = unsafe {
        CStr::from_bytes_with_nul_unchecked(bytes2)
    };
    println!("{:?}", bytes3);

Does this accomplish avoiding the runtime allocation and copying?

Upvotes: 0

Views: 859

Answers (2)

Chayim Friedman
Chayim Friedman

Reputation: 71380

Since Rust 1.72.0, CStr::from_bytes_with_nul() is const-stable, so prefer to use it:

macro_rules! include_cstr {
    ( $path:literal $(,)? ) => {{
        // Use a constant to force the verification to run at compile time.
        const VALUE: &'static ::core::ffi::CStr =
            match ::core::ffi::CStr::from_bytes_with_nul(concat!(include_str!($path), "\0").as_bytes()) {
                Ok(value) => value,
                Err(_) => panic!(concat!("interior NUL byte(s) in `", $path, "`")),
            };
        VALUE
    }};
}

let bytes = include_cstr!("triangle.vertex_shader");

Old Answer:

Your code is unsound. It fails to verify there are no interior NUL bytes.

You can use the following function to validate the string (at compile time, with no runtime cost):

pub const fn to_cstr(s: &str) -> &CStr {
    let bytes = s.as_bytes();
    let mut i = 0;
    while i < (bytes.len() - 1) {
        assert!(bytes[i] != b'\0', "interior byte cannot be NUL");
        i += 1;
    }
    assert!(bytes[bytes.len() - 1] == b'\0', "last byte must be NUL");
    // SAFETY: We verified there are no interior NULs and that the string ends with NUL.
    unsafe { CStr::from_bytes_with_nul_unchecked(bytes) }
}

Wrap it in a little nice macro:

macro_rules! include_cstr {
    ( $path:literal $(,)? ) => {{
        // Use a constant to force the verification to run at compile time.
        const VALUE: &'static ::core::ffi::CStr = $crate::to_cstr(concat!(include_str!($path), "\0"));
        VALUE
    }};
}

Then use it:

let bytes = include_cstr!("triangle.vertex_shader");

If there are interior NUL bytes the code will fail to compile.

Upvotes: 4

Locke
Locke

Reputation: 8982

Yes that should work as intended. If you want you can even bundle it into a simple macro.

macro_rules! include_cstr {
    ($file:expr) => {{
        // Create as explicit constant to force from_bytes_with_nul_unchecked to
        // perform compile time saftey checks.
        const CSTR: &'static ::std::ffi::CStr = unsafe {
            let input = concat!($file, "\0");
            ::std::ffi::CStr::from_bytes_with_nul_unchecked(input.as_bytes())
        };
        
        CSTR
    }};
}

const VERTEX_SHADER: &'static CStr = include_cstr!("shaders/vert.glsl");
const FRAGMENT_SHADER: &'static CStr = include_cstr!("shaders/frag.glsl");

Upvotes: -2

Related Questions