CODEWITHSUNDEEP
rustrakuffi
librasteve
librasteve

Reputation: 7581

Raku NativeCall to Rust FFI CArray Size Limitations - SegFault

Below is a Minimum(?) Reproducible Example of my code. It is a first pass at stress testing / benchmarking the WIP Raku Module Dan::Polars.

In Rust, I make a libmre.so with this code

  1 use libc::c_char;
  2 use libc::size_t;
  3 use std::slice;
  4 use std::ffi::*; //{CStr, CString,}
  5 
  6 //  Container
  7 
  8 pub struct VecC {
  9     ve: Vec::<String>,
 10 }   
 11 
 12 impl VecC {
 13     fn new(data: Vec::<String>) -> VecC               
 14     {   
 15         VecC {
 16             ve: data,
 17         }   
 18     }   
 19     
 20     fn show(&self) {
 21         println!{"{:?}", self.ve};
 22     }   
 23 }   
 24     
 25 #[no_mangle]
 26 pub extern "C" fn ve_new_str(ptr: *const *const c_char, len: size_t)                        
 27     -> *mut VecC {    
 28     
 29     let mut ve_data = Vec::<String>::new();
 30     unsafe {
 31         assert!(!ptr.is_null());
 32         
 33         for item in slice::from_raw_parts(ptr, len as usize) {
 34             ve_data.push(CStr::from_ptr(*item).to_string_lossy().into_owned());
 35         };  
 36     };  
 37     
 38     Box::into_raw(Box::new(VecC::new(ve_data)))
 39 }   
 40 
 41 #[no_mangle]
 42 pub extern "C" fn ve_show(ptr: *mut VecC) {
 43     let ve_c = unsafe {
 44         assert!(!ptr.is_null());
 45         &mut *ptr
 46     };  
 47     
 48     ve_c.show();
 49 }

and this Cargo.toml

  1 [package]
  2 name = "mre"
  3 version = "0.1.0"
  4 edition = "2021"
  5 
  6 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
  7 
  8 [dependencies]
  9 libc = "0.2.126"
 10 
 11 [lib]
 12 name = "mre"
 13 path = "src/lib.rs"
 14 crate-type = ["cdylib"]

In Raku, I consume libmre.so like this

  1 #!/usr/bin/env raku
  2 use lib '../lib';
  3 
  4 use NativeCall;
  5 
  6 #my $output;    #mre tried to move decl to here to avoid going out of scope
  7 sub carray( $dtype, @items ) {
  8     my $output := CArray[$dtype].new();
  9     loop ( my $i = 0; $i < @items; $i++ ) {
 10         $output[$i] = @items[$i]
 11     }
 12     say $output;
 13     $output
 14 }   
 15     
 16 ### Container Classes that interface to Rust lib.rs ###
 17 
 18 constant $n-path    = '../mre/target/debug/mre';
 19 
 20 class VecC is repr('CPointer') is export {
 21     sub ve_new_str(CArray[Str],size_t) returns VecC is native($n-path) { * }
 22     sub ve_show(VecC) is native($n-path) { * }
 23     
 24     method new(@data) { 
 25         ve_new_str(carray(Str, @data), @data.elems );
 26     }   
 27     
 28     method show {
 29         ve_show(self)
 30     }   
 31 }   
 32 
 33 my \N = 100;   #should be 2e9  #fails between 30 and 100
 34 my \K = 100;   
 35 
 36 sub randChar(\f, \numGrp, \N) {
 37     my @things = [sprintf(f, $_) for 1..numGrp];
 38     @things[[1..numGrp].roll(N)];
 39 }   
 40 
 41 my @data = [randChar("id%03d", K, N)];
 42 
 43 my $vec = VecC.new( @data );
 44 $vec.show;

When \N is <30 this runs fine with output like this:

NativeCall::Types::CArray[Str].new
["id098", "id035", "id024", "id067", "id051", "id025", "id024", "id092", "id044", "id042", "id033", "id004", "id100", "id091", "id087", "id059", "id031", "id063", "id019", "id035"]

When \N is > 50 however, I get:

NativeCall::Types::CArray[Str].new
Segmentation fault (core dumped)

This is:

Welcome to Rakudo™ v2022.04.
Implementing the Raku® Programming Language v6.d.
Built on MoarVM version 2022.04.
on ubuntu

Since the benchmark calls for \N to be 2e9, I could use some help to try and resolve this.

You are welcome to use p6steve/raku-dan:polars-2022.02-arm64 (or -amd64) on Docker Hub if you would like to try this at home. Don't forget to go cargo build first time. This includes RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | bash -s -- -y

Upvotes: 6

Views: 164

Answers (1)

gmoshkin
gmoshkin

Reputation: 1135

There's a off by one bug in randChar function

sub randChar(\f, \numGrp, \N) {
    my @things = [sprintf(f, $_) for 1..numGrp];     
    @things[[1..numGrp].roll(N)];
}

You're indexing the @things array with indexes from 1 to numGrp, but the biggest index of @things is numGrp - 1. So sometimes there's a (Any) instead of a string in one (or more) of the elements of the returned array.

What you want is this:

sub randChar(\f, \numGrp, \N) {
    my @things = [sprintf(f, $_) for 1..numGrp];     
    @things.roll(N); # call roll directly on @things
}

Upvotes: 7

Related Questions