SWIG and polymorphic C# using %typemap

Question

I would like to support downcasting in a SWIG-generated C# project.

I have a series of C++ std::shared_ptr-wrapped class templates that inherit from a common base. Any C++ method that returns a base class (IBasePtr) in C++ code results in a generated method that returns a concrete IBase object, which has no relation to the object I am actually trying to get. The blog post here deals with this exact problem by inserting custom code to perform a downcast based on object type metadata.

C++ (simplified for the purpose of illustration):

IBase.h:

namespace MyLib
{
    enum DataTypes
    {
        Float32,
        Float64,
        Integer32
    };

    typedef std::tr1::shared_ptr<IBase> IBasePtr;

    class IBase
    {
    public:
        virtual ~IBase() {}

        DataTypes DataType() const = 0;
    };
}

CDerived.h:

#include "IBase.h"

namespace MyLib
{
    template <class T>
    class CDerived : public IBase
    {
    public:
        CDerived(const DataTypes dataType)
        :
        m_dataType(dataType)
        {}

        DataTypes DataType() const
        {
            return m_dataType;
        }

    private:
        DataTypes m_dataType;
    };
}

CCaller.h:

#include "IBase.h"

namespace MyLib
{
    class CCaller
    {
    public:
        IBasePtr GetFloatObject()
        {
            //My code doesn't really do this - type identification is handled more elegantly, it's just to illustrate.
            base = IBasePtr(new CDerived<float>(Float32));
            return base;
        }

        IBasePtr GetDoubleObject()
        {
            //My code doesn't really do this - type identification is handled more elegantly, it's just to illustrate.
            base = IBasePtr(new CDerived<double>(Float64));
            return base;
        }
    private:
        IBasePtr base;
    };
}

SWIG interface:

%module SwigWrapper

%include "typemaps.i"
%include <cpointer.i>

#define SWIG_SHARED_PTR_SUBNAMESPACE tr1
%include <std_shared_ptr.i>

%shared_ptr(MyLib::IBase) 
%shared_ptr(MyLib::CDerived< float >)
%shared_ptr(MyLib::CDerived< double >)
%shared_ptr(MyLib::CDerived< int >)

%typemap(ctype, out="void *") MyLib::IBasePtr &OUTPUT "MyLib::IBasePtr *"
%typemap(imtype, out="IntPtr") MyLib::IBasePtr &OUTPUT "out IBase"
%typemap(cstype, out="$csclassname") MyLib::IBasePtr &OUTPUT "out IBase"
%typemap(csin) MyLib::IBasePtr &OUTPUT "out $csinput"
%typemap(in) MyLib::IBasePtr &OUTPUT

%{ $1 = ($1_ltype)$input; %}

%apply MyLib::IBasePtr &OUTPUT { MyLib::IBasePtr & base };

%{
#include "IBase.h"
#include "CDerived.h"
#include "CCaller.h"
using namespace std;
using namespace MyLib;
%}

namespace MyLib
{
    typedef std::tr1::shared_ptr<IBase> IBasePtr;

    %template (CDerivedFloat) CDerived<float>;
    %template (CDerivedDouble) CDerived<double>;
    %template (CDerivedInt) CDerived<int>;
}

%typemap(csout, excode=SWIGEXCODE)
IBase
IBasePtr
MyLib::IBase,
MyLib::IBasePtr
{
    IntPtr cPtr = $imcall;
    $csclassname ret = ($csclassname) $modulePINVOKE.InstantiateConcreteClass(cPtr, $owner);$excode
    return ret;
}

%pragma(csharp) imclasscode=%{
    public static IBase InstantiateConcreteClass(IntPtr cPtr, bool owner)
    {
        IBase ret = null;
        if (cPtr == IntPtr.Zero)
        {
            return ret;
        }

        int dataType = SwigWrapperPINVOKE.IBase_DataType(new HandleRef(null, cPtr));
        DataTypes dt = (DataTypes)dataType;

        switch (dt)
        {
            case DataTypes.Float32:
                ret = new CDerivedFloat(cPtr, owner);
                break;
            case DataTypes.Float64:
                ret = new CDerivedDouble(cPtr, owner);
                break;
            case DataTypes.Integer32:
                ret = new CDerivedInt(cPtr, owner);
                break;
            default:
                System.Diagnostics.Debug.Assert(false,
                String.Format("Encountered type '{0}' that is not a supported MyLib concrete class", dataType.ToString()));
                break;
        }   
        return ret;
    }
%}

The part I am struggling with is the use of SWIG's %typemap command. %typemap is intended to instruct SWIG to map input and target types, in my case via the code to perform an explicit conversion. The method InstantiateConcreteClass is generated but there are no references to it.

Is there a vital step I am missing? I wondered whether the was some additional complication due to the use of shared_ptr in native code, but I don't think this is the case.

Answer 1

Got this working with the help of Flexo's example above. Use of %newobject is essential here - in my original question there the lifetime of the derived objects was not being correctly managed.

I needed to make one minor change - the namespace name needed to be added to the typemap:

%typemap(csout, excode=SWIGEXCODE) MyLib::IBasePtr { // Need the fully-qualified name incl. namespace
    IntPtr cPtr = $imcall;
    var ret = $imclassname.make(cPtr, $owner);$excode // 3
    return ret;
}

It was not necessary to make the changes suggested after point 6.

Answer 2

The problem with your example seems to be that you've written typemaps for inputs, but that doesn't seem to make sense in and of itself, because the important part is getting the type right when creating things, not using them as input. As far as output arguments go the second half of this answer addresses that, but there are errors with using your typemaps for arguments too.

I've simplified your example fractionally and made it complete and working. The main thing I had to add that was missing was a 'factory' function that creates derived instances, but returns them as the base type. (If you just create them with new directly then this isn't needed).

I merged your header files and implemented an inline factory as test.h:

#include <memory>

enum DataTypes {
    Float32,
    Float64,
    Integer32
};

class IBase;

typedef std::shared_ptr<IBase> IBasePtr;

class IBase {
public:
    virtual ~IBase() {}
    virtual DataTypes DataType() const = 0;
};

template <typename T> struct DataTypesLookup;
template <> struct DataTypesLookup<float> { enum { value = Float32 }; };
template <> struct DataTypesLookup<double> { enum { value = Float64 }; };
template <> struct DataTypesLookup<int> { enum { value = Integer32 }; };

template <class T>
class CDerived : public IBase {
public:
    CDerived() : m_dataType(static_cast<DataTypes>(DataTypesLookup<T>::value)) {}

    DataTypes DataType() const {
        return m_dataType;
    }
private:
    const DataTypes m_dataType;
};

inline IBasePtr factory(const DataTypes type) {
    switch(type) {
    case Integer32:
        return std::make_shared<CDerived<int>>();
    case Float32:
        return std::make_shared<CDerived<float>>();
    case Float64:
        return std::make_shared<CDerived<double>>();
    }
    return IBasePtr();
}

The main changes here being the addition of some template meta programming to allow IBase to lookup the correct value of DataType from just the T template parameter and changing DataType to be const. I did this because it doesn't make sense to let CDerived instances to lie about their type - it's set exactly once and isn't something that should be exposed any further.

Given this I can then write some C# that shows how I intend to use it after wrapping:

using System;

public class HelloWorld {
    static public void Main() {
        var result = test.factory(DataTypes.Float32);
        Type type = result.GetType();
        Console.WriteLine(type.FullName);
        result = test.factory(DataTypes.Integer32);
        type = result.GetType();
        Console.WriteLine(type.FullName);
    }
}

Essentially if my typemaps are working we will have used the DataType member to transparently make test.factory return a C# proxy that matches the derived C++ type rather than a proxy that knows nothing more than the base type.

Note also that here because we have the factory we could also have modified the wrapping of that to use the input arguments to determine the output type, but this is less generic than using the DataType on the output. (For the factory approach we'd have to write per-function rather than per-type code for the correct wrapping).

We can write a SWIG interface for this example, which is substantially similar to yours and the blog post referenced but with a few changes:

%module test

%{
#include "test.h"
%}

%include <std_shared_ptr.i>

%shared_ptr(IBase) 
%shared_ptr(CDerived<float>)
%shared_ptr(CDerived<double>)
%shared_ptr(CDerived<int>)

%newobject factory; // 1

%typemap(csout, excode=SWIGEXCODE) IBasePtr { // 2
    IntPtr cPtr = $imcall;
    var ret = $imclassname.make(cPtr, $owner);$excode // 3
    return ret;
}

%include "test.h" // 4

%template (CDerivedFloat) CDerived<float>;
%template (CDerivedDouble) CDerived<double>;
%template (CDerivedInt) CDerived<int>;

%pragma(csharp) imclasscode=%{
    public static IBase make(IntPtr cPtr, bool owner) {
        IBase ret = null;
        if (IntPtr.Zero == cPtr) return ret;

        ret = new IBase(cPtr, false); // 5
        switch(ret.DataType()) {
            case DataTypes.Float32:
                ret = new CDerivedFloat(cPtr, owner);
                break;
            case DataTypes.Float64:
                ret = new CDerivedDouble(cPtr, owner);
                break;
            case DataTypes.Integer32:
                ret = new CDerivedInt(cPtr, owner);
                break;
            default:
                if (owner) ret = new IBase(cPtr, owner); // 6
                break;
        };
        return ret;
    }
%}

There are 6 notable changes highlighted via comments in that typemap:

1. We've told SWIG that the objects returned by factory are new, i.e. there is a transfer of ownership from C++ to C#. (This causes the owner boolean to get set correctly)
2. My typemap is a csout typemap and that's the only one needed.
3. Compared to the tutorial you linked I used $imclassname, which expands to $modulePINVOKE or equivalent correctly always.
4. I used %include with my header file directly to avoid repeating myself lots unnecessarily.
5. Rather than touch the inner workings of the wrapper I created a temporary instance of IBase directly that allowed me to access the enum value in a much cleaner way. The temporary instance has ownership set false that means we never incorrectly delete the underlying C++ instance when disposing of it.
6. I chose to let the default path through the switch statement return an IBase instance with no knowledge of the derived type if it couldn't be figured out for some reason.

---

Based on what you showed in your question it actually looks like what you're mostly struggling with is output reference arguments. Without the shared_ptr angle this wouldn't work at all. The simplest solution for wrapping this is to use %inline or %extend within SWIG to write an alternative version of the function to be used that doesn't pass output via reference arguments.

We can however make this work naturally on the C# side too, with some more typemaps. You're on the right track with the OUTPUT and %apply style typemaps you've shown, however I don't think you've got them quite right. I've extended my example to cover this as well.

First, although I don't much like using functions like this I added factory2 to test.h:

inline bool factory2(const DataTypes type, IBasePtr& result) {
    try {
        result = factory(type);
        return true;
    }
    catch (...) {
        return false;
    }
}

The key thing to note here is that by the time we call factory2 we must have a valid reference to an IBasePtr (std::shared_ptr<IBase>), even if that shared_ptr is null. Since you're using out instead of ref in you're C# we'll need to arrange to make a temporary C++ std::shared_ptr before the call actually happens. Once the call happens we want to pass this back to the make static function we wrote for the simpler case earlier.

We're going to have to look fairly closely at how SWIG handles smart pointers to make this all work.

So secondly my SWIG interface ended up adding:

%typemap(cstype) IBasePtr &OUTPUT "out $typemap(cstype,$1_ltype)"
%typemap(imtype) IBasePtr &OUTPUT "out IntPtr" // 1
// 2:
%typemap(csin,pre="    IntPtr temp$csinput = IntPtr.Zero;",
              post="    $csinput=$imclassname.make(temp$csinput,true);") 
    IBasePtr &OUTPUT "out temp$csinput" 
// 3:
%typemap(in) IBasePtr &OUTPUT {
    $1 = new $*1_ltype;
    *static_cast<intptr_t*>($input) = reinterpret_cast<intptr_t>($1);
}

%apply IBasePtr &OUTPUT { IBasePtr& result }

Before the %include of the simple case.

The main things this does are:

1. Change the intermediate function to accept an IntPtr by reference for output. This will eventually hold the value that we want to pass in to make that is itself a pointer to a std::shared_ptr.
2. For the csin typemap we're going to arrange to create a temporary IntPtr and use that for the intermediate call. After the intermediate call has happened we then need to pass the output into make and assign the resulting IBase instance to our output parameter.
3. When we call the real C++ function we need to construct a shared_ptr for it to bind to the reference when we make the call. We also store the address of the shared_ptr into our output parameter so that C# code can pick it up and work with it later.

This is now sufficient to solve our problem. I added the following code to my original test case:

    IBase result2;
    test.factory2(DataTypes.Float64, out result2);
    Console.WriteLine(result2.GetType().FullName);

---

A word of caution: this is the biggest chunk of C# code I've ever written. I tested it all on Linux with Mono using:

swig -c++ -Wall -csharp test.i && mcs -g hello.cs *.cs && g++ -std=c++11 -Wall -Wextra -shared -o libtest.so test_wrap.cxx
warning CS8029: Compatibility: Use -debug option instead of -g or --debug
warning CS2002: Source file `hello.cs' specified multiple times

Which when run gave:

CDerivedFloat
CDerivedInt
CDerivedDouble

and I think the generated marshalling is correct but you should verify it yourself.