What is typemap in SWIG (java c++)?

Question

I am trying to understand the SWIG (java c++ binding) but the online documentation seems pretty clumsy and badly organized. Went through few online links but none seem to explain in a naive friendly way. Eg. a) Why do we need typemaps? b) what does below means (please don't give me the documentation wording)

%typemap(jtype)
%typemap(jstype) 
%typemap(jni)
%typemap(javain)

c) What is the Director

d) is there any block diagram to see the SWIG and typemaps in action for java c++? I can't find a good concise tutorial for typemaps and the swig documentation is so jumpy that I can't keep track of things.

Answer 1

Why do we need typemaps?

Although the SWIG documentation is usually pretty solid you're right that it's hard to pick up the bigger picture from it.

I find it handy to thing about like this; you can approach SWIG from two different ways:

1. If you just want to wrap some C (or C++) code and have it all "just work, like in C" then all the typemaps you want are all supplied in the provided library and you won't need to worry about typemaps particularly. If you don't care to hand write JNI then skip typemaps and work with what's provided already
2. If you already understand JNI (or Python's C API for Python etc.) then this is where SWIG and typemaps really start to shine.

What is a typemap?

If you've written a JNI interface by hand (I'll keep using that for now, but this applies for pretty much any language you may care to target) one of the things you'll quickly find yourself doing is repeating chunks of code over and over. This happens because usually in any moderately sized library you care to wrap functions will share argument types and return types, patterns for error handling etc.

This is where typemaps come in. At the highest level they're fragments of code that are applied to build up the wrapping of your library. They're matched on the types involved and specify a mapping from something in C (or C++) to an equivalent in Java.

You can think of typemaps as something of a hybrid between Macros in C and templates in C++ - they're pretty simple manipulation of text without a huge deal of understanding of the language itself, but they are type aware and can be specialised, overridden etc. as you might with a C++ template.

What does that actually mean?

In practice the exact typemaps you'll care about in your SWIG interface depend on what language you're targeting - in the examples you showed they're all for something targeting JNI. There's plenty of this in the library of typemaps SWIG ships with, but some of those are more sophisticated than we really need to understand typemaps at the fundamental level.

As mentioned previously typemaps are mappings from the use of a C type to an appropriate wrapping in the target language. However since most languages you'd target have a fair amount of complexity to writing an interface by hand and SWIG needs to produce both Java and some C for you to compile for your interface there are multiple typemaps per C type (and per use of it, e.g. input, output, structure member).

What are these typemaps?

To answer your specific question about what each typemap in your list is used for let's dig a little deeper and see how to work this out for ourselves.

When I'm trying to figure something out about SWIG I usually use a combination of two things to help me do that. Firstly if you run SWIG with -debug-tmsearch as an extra argument then it tells you exactly what typemaps it's looking at, as well as which ones actually match in the end.

As an example the following interface:

%module test

MYOUTTYPE myfunction(MYINTYPE a);

When we run swig -java -debug-tmsearch test.i | grep Searching we get the following output:

eg1.i:3: Searching for a suitable 'jni' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'jtype' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'jstype' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'jni' typemap for: MYOUTTYPE myfunction
eg1.i:3: Searching for a suitable 'jtype' typemap for: MYOUTTYPE myfunction
eg1.i:3: Searching for a suitable 'default' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'arginit' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'in' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'typecheck' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'argout' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'check' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'freearg' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'except' typemap for: MYOUTTYPE myfunction
eg1.i:3: Searching for a suitable 'out' typemap for: MYOUTTYPE myfunction
eg1.i:3: Searching for a suitable 'ret' typemap for: MYOUTTYPE myfunction
eg1.i:3: Searching for a suitable 'jstype' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'javain' typemap for: MYINTYPE a
eg1.i:3: Searching for a suitable 'jstype' typemap for: MYOUTTYPE myfunction
eg1.i:3: Searching for a suitable 'javaout' typemap for: MYOUTTYPE myfunction
eg1.i:3: Searching for a suitable 'javabase' typemap for: MYOUTTYPE *
eg1.i:3: Searching for a suitable 'javainterfaces' typemap for: MYOUTTYPE *
eg1.i:3: Searching for a suitable 'javacode' typemap for: MYOUTTYPE *
eg1.i:3: Searching for a suitable 'javabody' typemap for: MYOUTTYPE *
eg1.i:3: Searching for a suitable 'javaclassmodifiers' typemap for: MYOUTTYPE *
eg1.i:3: Searching for a suitable 'javaimports' typemap for: MYOUTTYPE *
eg1.i:3: Searching for a suitable 'javabase' typemap for: MYINTYPE *
eg1.i:3: Searching for a suitable 'javainterfaces' typemap for: MYINTYPE *
eg1.i:3: Searching for a suitable 'javacode' typemap for: MYINTYPE *
eg1.i:3: Searching for a suitable 'javabody' typemap for: MYINTYPE *
eg1.i:3: Searching for a suitable 'javaclassmodifiers' typemap for: MYINTYPE *
eg1.i:3: Searching for a suitable 'javaimports' typemap for: MYINTYPE *

So we can see exactly which typemaps SWIG wants to consider in generating code for our dummy interface. Your list had the main ones here, in, javaout and out are probably the most notable ones missing.

This is often very helpful, and gives us enough to work with for my second trick for understanding things: Secondly you can use placeholder text for a typemap and see where it shows up.

For example if we make the following interface file:

%module test

%typemap(jtype) MYINTYPE "MYIN_JTYPE"
%typemap(jstype) MYINTYPE "MYIN_JSTYPE"
%typemap(jni) MYINTYPE "MYIN_JNI"
%typemap(javain) MYINTYPE "MYIN_JAVAIN"
%typemap(in) MYINTYPE "MYIN_IN"

MYOUTTYPE myfunction(MYINTYPE a);

Then we can do something like this after running SWIG:

$ grep MYIN *
eg2.i:%typemap(jtype) MYINTYPE "MYIN_JTYPE"
eg2.i:%typemap(jstype) MYINTYPE "MYIN_JSTYPE"
eg2.i:%typemap(jni) MYINTYPE "MYIN_JNI"
eg2.i:%typemap(javain) MYINTYPE "MYIN_JAVAIN"
eg2.i:%typemap(in) MYINTYPE "MYIN_IN"
eg2.i:MYOUTTYPE myfunction(MYINTYPE a);
eg2_wrap.c:SWIGEXPORT jlong JNICALL Java_testJNI_myfunction(JNIEnv *jenv, jclass jcls, MYIN_JNI jarg1) {
eg2_wrap.c:  MYINTYPE arg1 ;
eg2_wrap.c:  MYIN_IN
test.java:  public static SWIGTYPE_p_MYOUTTYPE myfunction(MYIN_JSTYPE a) {
test.java:    return new SWIGTYPE_p_MYOUTTYPE(testJNI.myfunction(MYIN_JAVAIN), true);
testJNI.java:  public final static native long myfunction(MYIN_JTYPE jarg1);

And from that we can begin to understand the context in which each of those typemaps is being applied in the generated code. (Remember that SWIG is basically concatenating the appropriate typemaps together to build this output). So roughly:

• jtype - gets used everywhere we need to figure out a corresponding Java type for a C one, in places where the type will actually cross between Java and C. So this is basically for the raw JNI functions.
• jstype - this is the type we want to present to Java users of our library. It needs to "match" with the jtype somewhat in that they will be paired together. Sometimes they'll be the same, but for example if you want to pass a pointer around then the jstype will be a Java proxy object, but the jtype will be a long. SWIG does have defaults for both of these for most cases though, so often you don't need to touch those.
• jni - this typemap is the type used for the corresponding JNI function - it's the JNI type equivalent of the jtype, typically it'll be jlong or jobject, again SWIG has defaults here which mean you often don't need to touch it. This is a C type rather than a Java type unlike the previous types.
• javain - this typemap is Java code rather than a type specifically. It's used in the generated proxy to convert between the exposed jstype and the jtype. That can be reaching into the proxy object and pulling a long which represents a C pointer out of it, or otherwise packing some data somehow, but it can also be just passing an object straight through too.
• in - this is some code again, this time in C that's responsible for converting from the jni type into the actual real type needed for the C function call. It shows up in the _wrap.c file that's generated instead of Java unlike most of the other examples.

I find most of these are easiest to understand in the context of the generated code. Docs and words are great and everything, but from my experience it's hard to beat running the tool, reading what's generated and seeing where something you write in a given typemap shows up. (And if it doesn't show up the debug-tmsearch argument can help you understand why).

Directors

Directors are some SWIG logic to handle polymorphism, i.e. virtual functions and the complexity that comes from crossing language boundaries and expecting this to work. If you only write C this doesn't matter, likewise C++ if you don't use the virtual keyword.

As above there's some extra typemaps involved and they show up in extra generated code, but it's completely explorable with the two techniques I suggested above.