CODEWITHSUNDEEP
cjulia
Jonas Hagmar
Jonas Hagmar

Reputation: 51

Constructing and returning Julia types in C

How would you construct and return a Julia Expr, say :(x + 2), from a C function called with ccall from Julia, preferably so that it can be garbage collected by Julia?

Edit:

As an edit was suggested with some additional details, here they are. To begin with, the question is not to generate the Expr in one go, but rater to be able to construct arbitrary Exprs and the like incremenally, see below.

I need a parser for a project in Julia. The contents of the parser input is in part algebraic expressions, and I would like to have the equivalent Julia Expr for these in the end. As there seems to be no parser generators with Julia targets, I was thinking about making the parser in C++ (which is convenient since I need to use the same parser grammar for C++ code in the same project anyway), either with flex/bison or ANTLR. So the question is really how you build the AST in Julia from the C++ parser. I have some ideas for solving this:

  1. Have a bunch of callbacks in Julia that the C++ code can call (e.g. push literal/add etc. for algebraic expressions) and that Julia manages the allocation of the AST nodes. Or maybe just calling into Symbol(), Expr() and so on, plus more specialized callbacks for structures specific to the problem. Even if I am very new to Julia, I think I could pull that off.
  2. Let the C++ parser generate an AST of plain structs that the Julia code can traverse to build the equivalent Julia structures. A bit of a messy solution, but maybe.
  3. Have the C++ code allocate and return the Julia AST directly, which I think, if it would be possible, amounts to an AST of jl_type_t and friends. The big question is how such a structure is constructed. I have not found any documentation except the .h files, which are not that well commented. And I also doubt that memory allocated in the C++ code can be freed (garbage collected) in Julia, and I wonder how you could solve that.

Upvotes: 0

Views: 315

Answers (1)

Jonas Hagmar
Jonas Hagmar

Reputation: 51

Answering my own question, I think that going for strategy 1 as stated in the question is the easiest and most robust way to solve this. Strategy 2 is possible, but that involves having mirrored ASTs in C and Julia, which makes the solution harder to maintain. Strategy 3, if even possible, requires in-depth knowledge about how the Julia type system works under the hood, and essentially amounts to duplicating functionality from the Julia C source code.

The only (minor) nuisance with my solution is that I have not been able to get a pointer to vararg Expr using @cfunction. So in the context of algebraic expressions, you would need separate function pointers to construct unary and binary operations, for example. If anyone has more success with this, let me know.

Code

expr.c

// struct to gather neccessary callbacks
typedef struct {
  void* (*symbol_callback)(char const *); // Symbol(Cstring)
  void* (*int_callback)(int); // Int(Int) - boxing an int
  void* (*expr4_callback)(void *, void*, void*, void*); // Expr for binary op
} callbacks;

void *expr(callbacks *c) {
  return c->expr4_callback(
    c->symbol_callback("call"),
    c->symbol_callback("+"),
    c->symbol_callback("x"),
    c->int_callback(2)
  );
}

Makefile

CC=gcc

CFLAGS=-c -Wall -fPIC

SOURCES=expr.c
OBJECTS=$(SOURCES:.c=.o)

.c.o:
    $(CC) $(CFLAGS) $< -o $@

lib: $(OBJECTS)
    $(CC) -shared -fPIC -o libexpr.so $(OBJECTS)

clean:
    rm *.o *.so

Test.jl

# define a proxy function for Symbol(::String) that does the conversion from
# a C byte string
symbol_proxy(name::Ptr{UInt8})::Symbol = Symbol(unsafe_string(name))

struct Callbacks
    # pointer to Symbol(::String) proxy
    symbol::Ptr{Nothing}
    # pointer to Int(::Int) function
    int64::Ptr{Nothing}
    # pointer to Expr() function with four arguments (for binary op)
    expr4::Ptr{Nothing}
end

# get callbacks
c = Callbacks(
    @cfunction(symbol_proxy, Ref{Symbol}, (Ptr{UInt8},)),
    @cfunction(Int, Ref{Int64}, (Cint,)),
    @cfunction(Expr, Ref{Expr}, (Any,Any,Any,Any))
)

# call shared library to construct :(x+2)
e = ccall((:expr, "./libexpr"), Ref{Expr}, (Ref{Callbacks},), c)
dump(e)

Output

Compiling the shared library with make lib and running Test.jl produces the output

Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: Symbol +
    2: Symbol x
    3: Int64 2

Upvotes: 0

Related Questions