CODEWITHSUNDEEP
pythonc++swig
Mike Finch
Mike Finch

Reputation: 877

How to pass byte buffer from C++ to Python using SWIG?

I am using SWIG version 4.0.2 in a Windows Subsystem for Linux (WSL) Ubuntu distribution. The C++ class I want to wrap contains an array of bytes (i.e., each item in the array is of type uint8_t or unsigned char). The C++ class has a method whose output parameter is a double pointer, so that the caller can obtain a pointer to that array of bytes.

The simple C++ class to wrap is represented by the following header file.

example6.h
#pragma once

class Message
{
private:
  uint8_t * m_data;
  int m_size;

public:
  Message( void )
  : m_data( NULL )
  , m_size( 0 )
  {
    m_data = new uint8_t[3]();
    m_data[0] = 'A';
    m_data[1] = 'B';
    m_data[2] = 'C';
    m_size = 3;
  }

  virtual ~Message( void )
  {
    delete[] m_data;
  }

  int package( uint8_t** buf )
  {
    *buf = NULL;
    int size = 0;
    if ( m_data )
    {
      *buf = m_data;
      size = m_size;
    }

    return size;
  }
};

The SWIG interface file I use to wrap the C++ code is the following.

example6.swg
%module example6

%{
  #define SWIG_FILE_WITH_INIT

  #include "example6.h"
%}

%typemap( in, numinputs=0 ) ( uint8_t** buf )
{
  uint8_t** temp = 0x0;
  $1 = temp;
}

%typemap( argout ) ( uint8_t** buf )
{
  if ( *$1 )
  {
    $result = PyBytes_FromString( (char *)( *$1 ) );
    free( *$1 );
  }
}

%include "example6.h"

In the SWIG interface file, I am attempting to use appropriate %typemap(in) and %typemap(argout) directives, based on breadcrumbs from several examples. I have several concerns about them and do not know yet if they will work. However, those are not the first obstacle.

I am not certain what type of Python variable to provide as the argument for the Message.package() method when I call it. I think I want the variable to be of type bytes. Or, maybe I need additional %typemap directives to correctly convert the Python bytes variable to a C++ uint8_t** variable?

The following is a demonstration of the problem.

finch@laptop:~/work/swig_example$ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import example6
>>> m = example6.Message()
>>> b = bytes()
>>> b
b''
>>> m.package( b )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/finch/work/swig_example/example6.py", line 73, in package
    return _example6.Message_package(self, buf)
TypeError: in method 'Message_package', argument 2 of type 'uint8_t **'

The TypeError exception is raised in the SWIG-generated example6_wrap.cxx code. The exception happens well before it gets a chance to execute the C++ snippet that the %typemap(argout) directive above injects into it.

example6_wrap.cxx
SWIGINTERN PyObject *_wrap_Message_package(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
  PyObject *resultobj = 0;
  Message *arg1 = (Message *) 0 ;
  uint8_t **arg2 = (uint8_t **) 0 ;
  void *argp1 = 0 ;
  int res1 = 0 ;
  void *argp2 = 0 ;
  int res2 = 0 ;
  PyObject *swig_obj[2] ;
  int result;

  if (!SWIG_Python_UnpackTuple(args, "Message_package", 2, 2, swig_obj)) SWIG_fail;
  res1 = SWIG_ConvertPtr(swig_obj[0], &argp1,SWIGTYPE_p_Message, 0 |  0 );
  if (!SWIG_IsOK(res1)) {
    SWIG_exception_fail(SWIG_ArgError(res1), "in method '" "Message_package" "', argument " "1"" of type '" "Message *""'"); 
  }
  arg1 = reinterpret_cast< Message * >(argp1);
  res2 = SWIG_ConvertPtr(swig_obj[1], &argp2,SWIGTYPE_p_p_uint8_t, 0 |  0 );
  if (!SWIG_IsOK(res2)) {
    //*** Finch temp: This is where the TypeError exception will be raised.
    SWIG_exception_fail(SWIG_ArgError(res2), "in method '" "Message_package" "', argument " "2"" of type '" "uint8_t **""'"); 
  }
  arg2 = reinterpret_cast< uint8_t ** >(argp2);
  result = (int)(arg1)->package(arg2);
  resultobj = SWIG_From_int(static_cast< int >(result));

  //*** Finch temp: This is the snippet that the %typemap(argout) directive injects.
  {
    if ( *arg2 )
    {
      resultobj = PyBytes_FromString( (char *)( *arg2 ) );
      free( *arg2 );
    }
  }
  //***

  return resultobj;
fail:
  return NULL;
}

Problem 1

What typemap will help me convert a Python bytes variable to a C++ uint8_t** variable.

Problem 2

PyBytes_FromString() expects to be given a char const * variable. But the underlying message data in the real C++ library I am wrapping is uint8_t; I am stuck with that. I worry that blindly recasting will cause some byte values to be wrong. Is there another PyBytes_something() function I have not seen?

If not, is it appropriate for the %typemap(argout) directive to have a lot more involved code in it, such as iterating the byte buffer and manually copying it to ... something else?

I have been searching for and piecing together examples from several places, including:

Upvotes: 0

Views: 68

Answers (1)

Mike Finch
Mike Finch

Reputation: 877

I have a kludge that is sufficient for my needs. However, it is Python specific and does not solve the problem in general.

In the SWIG interface file, instead of relying on typemaps to automatically wrap the Message::package() method, I added an inline code insertion block, and within that a helper function that manually wraps the Message::package() method. The helper function returns a PyObject pointer which, in the Python environment, is correctly interpreted as a bytes object that contains a copy of the message data.

example6.swg
%module example6

...

%inline %{
  PyObject * Message_package_helper( Message * pMsg )
  {
    PyObject * pResult = 0x0;
    if ( pMsg )
    {
      uint8_t * pData;
      int size = pMsg->package( &pData );
      pResult = PyBytes_FromStringAndSize( reinterpret_cast<char *>( pData ), size );
    }

    return pResult;
  }
%}

...

The following is a demonstration that it works.

finch@laptop:~/work/swig_example$ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import example6
>>> m = example6.Message()
>>> b = example6.Message_package_helper( m )
>>> b
b'ABC'

Documentation I read about SWIG code insertion blocks, helper functions, and inlining:

Upvotes: 0

Related Questions