CODEWITHSUNDEEP
java-native-interfaceapache-arrow
Timothy Miller
Timothy Miller

Reputation: 11

How to create Apache Arrow vectors in Java, pass them to C++ code through JNI, read/write them in C++

I've been reading the Apache Arrow docs, and I've figured out how to use it in Java and C++. But what I'd like to do is offload some work to JNI (C/C++) code from Java, and the documentation (e.g. https://arrow.apache.org/docs/java/cdata.html) just doesn't seem to cover my use cases, and methods in the example (e.g. getMemoryAddress on IntVector) just don't seem to exist like they do in the examples. I want to start simple, so here's what I'd like to do:

Can anyone point me to an example or some tips on how to do this?

BTW, the examples also use JavaCPP instead of JNI. But I already have a bunch of JNI code in this project, and I'd rather not mix in another kind of bridge if it's not necessary.

Thanks.

I tried allocating IntVector objects in Java, but I can't tell which naive pointers I have to retrieve to pass to C++ to provide proper access to those vectors.

Upvotes: 1

Views: 560

Answers (1)

li.davidm
li.davidm

Reputation: 12116

JavaCPP is merely a convenience for the example, JNI is fine.

The C Data Interface is still what you want. When you say "get whatever native pointers I need": that is exactly what a struct ArrowArray is in the C Data Interface. Use the C Data Interface module in Java to export your Java arrays and get the address of a struct ArrowArray, and then pass that address to your C++ code via JNI. Then, use libarrow's C Data Interface implementation to import the arrays and work with them.

When the C++ side is done, it does the same thing: it exports the result vector and returns an address to Java via JNI; the Java code then imports the vector from that address.

Upvotes: 3

Related Questions