SWIG wrap for Python a void function with an output reference to smart pointer parameter

Question

Along the lines of this question, suppose we have a C++ struct S and a function makeS which creates an instance of S and assigns it to a shared pointer p. Here is a self-contained running example:

#include <iostream>
#include <memory>

struct S
{
    int x;
    
    S() { x = 0; std::cout << "S::S()\n"; }
 
    // Note: non-virtual destructor is OK here
    ~S() { std::cout << "S::~S()\n"; }
};
 
 
void makeS(std::shared_ptr<S> &p)
{
    p = std::make_shared<S>();
    p->x = 12;
}
 
int main()
{
    std::shared_ptr<S> p;
    makeS(p);
    std::cout << "x = " << p->x << "\n";
}

which outputs

S::S()
x = 12
S::~S()

How do we wrap void makeS(std::shared_ptr<S> &p) in SWIG for Python 3 so that, in Python, we can run

p = makeS()

and get a smart pointer to an instance of S? In other words, how do we write the Python-based %typemap for std::shared_ptr<S> & so that we can write something like

%apply std::shared_ptr<S> &OUTPUT { std::shared_ptr<S> & }

and not get an error from SWIG that there is no typemap defined?

UPDATE: With input from Mark Tolonen below, I can add more detail but I am still stuck. We have the following files:

Widget.h

#include <iostream>
#include <memory>

struct Widget
{
    int x;
    
    Widget() { x = 0; std::cout << "Widget::Widget()\n"; }
 
    // Note: non-virtual destructor is OK here
    ~Widget() { std::cout << "Widget::~Widget()\n"; }
};
 
void makeWidget(std::shared_ptr<Widget> &p);

Widget.cpp

#include "Widget.h"

void makeWidget(std::shared_ptr<Widget> &p)
  {
    p = std::make_shared<Widget>();
    p->x = 12;
}

main.cpp

#include "Widget.h"
 
int main()
{
    std::shared_ptr<Widget> p;
    makeWidget(p);
    std::cout << "x = " << p->x << "\n";
}

From here I can execute (on Ubuntu):

g++ -O2 -fPIC -c Widget.cpp
g++ -O2 -fPIC -c main.cpp
g++ Widget.o main.o -o widget
widget

and get an output from running widget of

Widget::Widget()
x = 12
Widget::~Widget()

Next we create a wrap

Widget.i

%module Widget

%include <typemaps.i> // for OUTPUT
%include <std_shared_ptr.i>
%shared_ptr(Widget);

%{
#include "Widget.h"
%}

// Declare an input typemap that suppresses requiring any input.
%typemap(in, numinputs=0) std::shared_ptr<Widget>& %{
%}

// Declare an output argument typemap updates the shared pointer,
// converts it to a Python object, and appends it to the return value.
%typemap(argout) std::shared_ptr<Widget>& %{

  try { 
    PyObject* obj = SWIG_NewPointerObj(&arg1, $descriptor(std::shared_ptr<Widget>*), 0);
    $result = SWIG_Python_AppendOutput($result, obj); }
  catch (...) {
    SWIG_fail;
  }

%}

%include "Widget.h"

Running

swig -c++ -python -o Widget_wrap.cpp Widget.i

will generate the following code in the new file

Widget_wrap.cpp:

SWIGINTERN PyObject *_wrap_makeWidget(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
  PyObject *resultobj = 0;
  std::shared_ptr< Widget > *arg1 = 0 ;
  
  
  
  if (!SWIG_Python_UnpackTuple(args, "makeWidget", 0, 0, 0)) SWIG_fail;
  makeWidget(*arg1);
  resultobj = SWIG_Py_Void();
  
  
  try {
    PyObject* obj = SWIG_NewPointerObj(&arg1, SWIGTYPE_p_std__shared_ptrT_Widget_t, 0);
    resultobj = SWIG_Python_AppendOutput(resultobj, obj); 
  }
  catch (...) {
    SWIG_fail;
  }
  
  
  return resultobj;
fail:
  return NULL;
}

Notice that the variable arg1 and call makeWidget(*arg1); were generated by Swig on its own and are not contained in the typemap in the .i file. After a prior pass, I saw them and that is why I use arg1 in the typemap. We compile and build the .so file:

 g++ -O2 -fPIC -I/home/catskills/anaconda3/envs/trader/include/python3.10 -c Widget_wrap.cpp
 g++ -shared *Widget.o Widget_wrap.o -o _Widget.so

Unfortunately then loading and running this in Python gives a seg fault:

$ python
Python 3.9.7 (default, Sep 16 2021, 13:09:58) 
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import Widget
>>> Widget.makeWidget()
Widget::Widget()
Segmentation fault (core dumped)

QUESTION: How do we get rid of the seg fault?

UPDATE 2: To answer this question, I modified the example to the more usual case of writing a function that returns a result, rather than a void function with a result out parameter. The resulting Widget_wrap.cpp gives some insight on how to wrap for the desired case. The modified case is:

Widget.h

#include <iostream>
#include <memory>

struct Widget
{
    int x;
    
    Widget() { x = 0; std::cout << "Widget::Widget()\n"; }
 
    // Note: non-virtual destructor is OK here
    ~Widget() { std::cout << "Widget::~Widget()\n"; }
};
 
std::shared_ptr<Widget> makeWidget();

Widget.cpp

#include "Widget.h"

std::shared_ptr<Widget> makeWidget()
{
    std::shared_ptr<Widget> p = std::make_shared<Widget>();
    p->x = 12;
    return p;
}

Widget.i

%module Widget

%include <std_shared_ptr.i>
%shared_ptr(Widget);

%{
#include "Widget.h"
%}

%include "Widget.h"

This produces:

Widget_wrap.cpp

SWIGINTERN PyObject *_wrap_makeWidget(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
  PyObject *resultobj = 0;
  std::shared_ptr< Widget > result;
  
  if (!SWIG_Python_UnpackTuple(args, "makeWidget", 0, 0, 0)) SWIG_fail;
  result = makeWidget();
  {
    std::shared_ptr<  Widget > *smartresult = result ? new std::shared_ptr<  Widget >(result) : 0;
    resultobj = SWIG_NewPointerObj(SWIG_as_voidptr(smartresult), SWIGTYPE_p_std__shared_ptrT_Widget_t, SWIG_POINTER_OWN);
  }
  return resultobj;
fail:
  return NULL;
}

As we can see there is quite a fancy treatment of the conversion of the shared pointer into a Swig pointer. If we then transplant that manually into the Widget_wrap.cpp of the original case, we get this:

SWIGINTERN PyObject *_wrap_makeWidget(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
  PyObject *resultobj = 0;
  std::shared_ptr< Widget > result;

  if (!SWIG_Python_UnpackTuple(args, "makeWidget", 0, 0, 0)) SWIG_fail;
  makeWidget(result);
  {
    std::shared_ptr<  Widget > *smartresult = result ? new std::shared_ptr<  Widget >(result) : 0;
    resultobj = SWIG_NewPointerObj(SWIG_as_voidptr(smartresult), SWIGTYPE_p_std__shared_ptrT_Widget_t, SWIG_POINTER_OWN);
  }
  return resultobj;
fail:
  return NULL;
}

If we build this and run it through this Python test program:

test.py

from Widget import *
x = makeWidget()
print(x)
y = makeWidget()
print(y)

We get a nice result:

$ python test.py
Widget::Widget()
<Widget.Widget; proxy of <Swig Object of type 'std::shared_ptr< Widget > *' at 0x7f05f6329f00> >
Widget::Widget()
<Widget.Widget; proxy of <Swig Object of type 'std::shared_ptr< Widget > *' at 0x7f05f61941e0> >
Widget::~Widget()
Widget::~Widget()

This leads to a final revised question (working on this next):

QUESTION: How do I write a typemap which gives me exactly the manually edited version of _wrap_makeWidget above, with no redundant calls to makeWidget?

Answer 1

Here is the final answer, with great help from Mark Tolenen's answer and updates. I am leaving the journey to this in the question unless moderators prefer me to compact the question in hindsight:

Widget.h

#include <iostream>
#include <memory>

struct Widget
{
    int x;
    
    Widget() { x = 0; std::cout << "Widget::Widget()\n"; }
 
    // Note: non-virtual destructor is OK here
    ~Widget() { std::cout << "Widget::~Widget()\n"; }
};
 
void makeWidget(std::shared_ptr<Widget> &p);

Widget.cpp

#include "Widget.h"

void makeWidget(std::shared_ptr<Widget> &p)
  {
    p = std::make_shared<Widget>();
    p->x = 12;
}

Widget.i

// Great thanks to Mark Tolonen
%module Widget

%include <typemaps.i> // for OUTPUT
%include <std_shared_ptr.i>
%shared_ptr(Widget);

%{
#include "Widget.h"
%}

// Declare an input typemap that suppresses requiring any input
// and creates a temporary object that can be referenced.
%typemap(in, numinputs=0) std::shared_ptr<Widget>& (std::shared_ptr<Widget> tmp) %{
    $1 = &tmp;
%}

// Declare an output argument typemap updates the shared pointer,
// converts it to a Python object, and appends it to the return value.
%typemap(argout) std::shared_ptr<Widget>& %{
  {
    std::shared_ptr<  Widget > *smartresult = tmp1 ? new std::shared_ptr<  Widget >(tmp1) : 0;
    resultobj = SWIG_NewPointerObj(SWIG_as_voidptr(smartresult), SWIGTYPE_p_std__shared_ptrT_Widget_t, SWIG_POINTER_OWN);
  }
%}

%include "Widget.h"

main.cpp

#include "Widget.h"
 
int main()
{
    std::shared_ptr<Widget> p;
    makeWidget(p);
    std::cout << "x = " << p->x << "\n";
}

test.py

from Widget import *
x = makeWidget()
print(x)
y = makeWidget()
print(y)

make.sh

#!/bin/bash -x
g++ -O2 -fPIC -c Widget.cpp
g++ -O2 -fPIC -c main.cpp
g++ -O2 -fPIC Widget.o main.o -o widget
widget
swig -c++ -python -o Widget_wrap.cpp Widget.i
g++ -O2 -fPIC -I/home/catskills/anaconda3/envs/trader/include/python3.10 -c Widget_wrap.cpp
g++ -shared Widget.o Widget_wrap.o -o _Widget.so
python test.py

Output of running make.sh

$ make.sh
+ g++ -O2 -fPIC -c Widget.cpp
+ g++ -O2 -fPIC -c main.cpp
+ g++ -O2 -fPIC Widget.o main.o -o widget
+ widget
Widget::Widget()
x = 12
Widget::~Widget()
+ swig -c++ -python -o Widget_wrap.cpp Widget.i
+ g++ -O2 -fPIC -I/home/catskills/anaconda3/envs/trader/include/python3.10 -c Widget_wrap.cpp
+ g++ -shared Widget.o Widget_wrap.o -o _Widget.so
+ python test.py
Widget::Widget()
<Widget.Widget; proxy of <Swig Object of type 'std::shared_ptr< Widget > *' at 0x7f25e870ef30> >
Widget::Widget()
<Widget.Widget; proxy of <Swig Object of type 'std::shared_ptr< Widget > *' at 0x7f25e85791e0> >
Widget::~Widget()
Widget::~Widget()

Answer 2

I don’t have a working computer at the moment to test a working example, but you need the following in the SWIG .i file before you %include your header defining the functions that use your shared pointer.

The OUTPUT typemap can only be applied to a few basic types. The typemaps below are my best guess without a computer to verify.

%module test

%include <std_shared_ptr.i>
%shared_ptr(Widget);

%{
#include "Widget.h"
%}

// Declare an input typemap that suppresses requiring any input
// and creates a temporary object that can be referenced.
%typemap(in, numinputs=0) std::shared_ptr<Widget>& (std::shared_ptr<Widget> tmp) %{
    $1 = tmp;
%}

// Declare an output argument typemap updates the shared pointer,
// converts it to a Python object, and appends it to the return value.
%typemap(argout) std::shared_ptr<Widget>& %{
    makeWidget($1);
    PyObject* obj = SWIG_NewPointerObj(&$1, $descriptor(std::shared_ptr<Widget>*), 0)
    $result = SWIG_Python_AppendOutput($result, obj);
%}

%include "Widget.h"