CODEWITHSUNDEEP
rustdllsegmentation-faultrust-cargodangling-pointer
TwiceUponATime
TwiceUponATime

Reputation: 3

Segfault in Rust when calling a function returned by a cdylib crate function

If I have a fn(&mut Box<fn()>) in a cdylib crate, and it sets the value of the Box to another function defined in the crate, I get a segmentation fault when calling the resulted function.

I have two crates to test this, lib and use-lib. use-lib uses libloading 0.8.6, but there are no other dependencies.

Here is my code in lib/src/lib.rs:

fn dangling() {
    println!("Hello from the returned function");
}

#[no_mangle]
pub extern "Rust" fn loaded_fn(func: &mut Box<fn()>) {
    **func = dangling;
}

Here is my code in use-lib/src/main.rs (I'm loading a DLL because I'm on Windows, I don't know if that affects the problem):

use libloading::{Library, Symbol};

fn default() {
    println!("function not modified");
}

fn main() {
    let mut dangling_fn: Box<fn()> = Box::new(default);

    unsafe {
        type LoadedFn = extern "Rust" fn(&mut Box<fn()>);

        let lib = Library::new("./lib.dll").unwrap_or_else(|err| {
            panic!("unable to load library: {}", err);
        });
        println!("loaded lib");
        
        let loaded_fn: Symbol<LoadedFn> = lib.get(b"loaded_fn").unwrap_or_else(|err| {
            panic!("unable to get symbol: {}", err);
        });
        println!("loaded function");

        loaded_fn(&mut dangling_fn);
        println!("called function");
    }

    println!("calling dangling function");
    dangling_fn();
}

This is the output I get:

loaded lib
loaded function
called function
calling dangling function
error: process didn't exit successfully: `target\debug\use-lib.exe` (exit code: 0xc0000005, STATUS_ACCESS_VIOLATION)

My diagnosis is that returned_fn is never loaded into use_lib, but how can I fix this?

Upvotes: 0

Views: 65

Answers (1)

cafce25
cafce25

Reputation: 27419

When lib goes out of scope, the library is unloaded, so your dangling_fn is truely dangling after the unsafe block. So you have to lift it out of the scope:

fn main() {
    let mut dangling_fn: Box<fn()> = Box::new(default);
    let lib;
    unsafe {
        // ...
        lib = Library::new("./lib.dll").unwrap_or_else(|err| {
            panic!("unable to load library: {}", err);
        });
        // ...
    }
}

Or maybe even better just wrap the parts that actually need unsafe and avoid creating such a big scope to begin with, this also has the added benefit of clearly marking just the unsafe calls:

use libloading::{Library, Symbol};

fn default() {
    println!("function not modified");
}

fn main() {
    let mut dangling_fn: Box<fn()> = Box::new(default);

    type LoadedFn = extern "Rust" fn(&mut Box<fn()>);

    let lib = unsafe { Library::new("./lib.dll") }.unwrap_or_else(|err| {
        panic!("unable to load library: {}", err);
    });
    println!("loaded lib");

    let loaded_fn: Symbol<LoadedFn> = unsafe { lib.get(b"loaded_fn") }.unwrap_or_else(|err| {
        panic!("unable to get symbol: {}", err);
    });
    println!("loaded function");
    loaded_fn(&mut dangling_fn);
    println!("called function");
    println!("calling dangling function");
    dangling_fn();
}

Upvotes: 3

Related Questions