CODEWITHSUNDEEP
c++pythonboostportabilityboost-python
eladidan
eladidan

Reputation: 2644

Boost Python portability concerns

I have a dll written in C++ that I want to export to Python for running regression and unit-testing (it's just easier to maintain and run the regression with Python). To this purpose I want to use Boost.Python to export the main API of the dll so it will be usable in Python. My assemblies look as follows:

  1. MyLibrary.dll //main API C++ library
  2. MyLibrary.pyd //a thin dll project containing only the BOOST_PYTHON_MODULE export definitions (dependant on MyLibrary.dll)
  3. ... //other C++ dll files that MyLibrary.dll is dependent on

I had some trouble getting MyLibrary.pyd to link but after digging through questions a bit (e.g. here) I realised I had to re-build boost while pointing b2.exe to my specific Python version. After which I was able to import and run my library from python (on my machine alone).

Technical data: I'm building the libraries with boost 1.51, Python 3.23 on Windows 7 x64 and MSVC-10.0 (my own projects are built from VS2010). The variant I'm using to link with boost is shared libraries, 64 address model, release accordingly with my own builds.

The problem is, when I try to import the library (built on my machine) on another machine, python complains: 

ImportError: DLL load failed: The specified procedure could not be found.

On the line import MyLibrary

Which begs the following questions:

  1. Is the MyLibrary.pyd I built on my machine "python-portable"? Meaning, will it work on other versions of Python besides 3.23 (the version I used to build boost.python with on my machine)?
  2. Does the user of MyLibrary.pyd have to re-build boost with his own version of python in order to be able to successfully import it?
  3. So far we've been using the pre-built boost installer for windows supplied by BoostPro. What version of Python is that build linked against and can I save my users the headache of building boost on their own if we simply decide to work with the "right" version of Python across the team (the version BoostPro linked against)?

Upvotes: 4

Views: 1322

Answers (1)

Wojciech Cierpucha
Wojciech Cierpucha

Reputation: 810

Take a look at PEP 384 at http://docs.python.org/3.2/whatsnew/3.2.html.

http://www.boost.org/doc/libs/1_52_0/libs/python/doc/news.html shows that there hasn't been any real progress lately so I doubt that Boost.Python supports or was at least tested with Py_LIMITED_API defined.

According to my experience with Python 2.x compatibility using both Boost.Python and PyCXX (I haven't worked with 3.x line yet):

  1. No it won't. Only micro version changes stay ABI portable.
  2. Not exactly. The user of a MyLibrary.pyd binary provided by you won't be able to load it using different major/minor python version. The build configuration of Boost she has does not matter. You need to have Boost.Python builds with every minor Python version you want to support. That includes separate builds for 32 and 64 bit Python installations.

My advice is to try to build Boost from source with Py_LIMITED_API defined. I do not guarantee that it will succeed, but it's worth a try.

If it fails ask your teammates to use the same Python version as you and of course a x64 bit Windows (since the .pyd is 64bit itself). Or even better setup a CI machine that will build your python module in every required configuration so that your clients will be able to choose a proper binary. Let your teammates build and use their own versions of MyLibrary.pyd for their local use only.

Upvotes: 3

Related Questions