Embed python in c++: choose python version

Question

I've been searching an answer to my question for quite a while but none of the ones that I have found seems to solve my problem.

I'm trying to embed Python within my C++ code with the functionalities provided by Python (Python.h, Py_xxx functions, etc.). However, I'm having troubles in getting my C++ program to call the right Python interpreter. Indeed, there exist several interpreters on my machine (which by the way is a Mac running OSX 10.7.5). I have the default version of Python preinstalled with the OS (ver 2.7.1) and I have another version installed by anaconda (ver 2.7.7). I need to use the version installed by anaconda because I need extra libraries available with anaconda that are not installed by default with OSX's Python.

My C++ code is as follows:

char* python_home_ = (char*) "/anaconda";
char* program_name_ = (char*) "/anaconda/bin/python2.7";

Py_SetPythonHome(python_home_);
Py_SetProgramName(program_name_);

Py_Initialize();

printf("python home: %s\n", Py_GetPythonHome());
printf("program name: %s\n", Py_GetProgramName());
printf("get path: %s\n", Py_GetPath());
printf("get prefix: %s\n", Py_GetPrefix());
printf("get exec prefix: %s\n", Py_GetExecPrefix());
printf("get prog full path: %s\n", Py_GetProgramFullPath());

PyRun_SimpleString("import sys");
printf("path: ");
PyRun_SimpleString("print sys.path");
printf("version: ");
PyRun_SimpleString("print sys.version");

And the result:

python home: /anaconda
program name: /anaconda/bin/python2.7
get path: /anaconda/lib/python27.zip:/anaconda/lib/python2.7/:/anaconda/lib/python2.7/plat-darwin:/anaconda/lib/python2.7/plat-mac:/anaconda/lib/python2.7/plat-mac/lib-scriptpackages:/anaconda/lib/python2.7/../../Extras/lib/python:/anaconda/lib/python2.7/lib-tk:/anaconda/lib/python2.7/lib-old:/anaconda/lib/python2.7/lib-dynload
get prefix: /anaconda
get exec prefix: /anaconda
get prog full path: /anaconda/bin/python2.7
path: ['/anaconda/lib/python2.7/site-packages/sphinxcontrib_googleanalytics-0.1dev_20140616-py2.7.egg', '/anaconda/lib/python27.zip', '/anaconda/lib/python2.7', '/anaconda/lib/python2.7/plat-darwin', '/anaconda/lib/python2.7/plat-mac', '/anaconda/lib/python2.7/plat-mac/lib-scriptpackages', '/anaconda/Extras/lib/python', '/anaconda/lib/python2.7/lib-tk', '/anaconda/lib/python2.7/lib-old', '/anaconda/lib/python2.7/lib-dynload', '/anaconda/lib/python2.7/site-packages', '/anaconda/lib/python2.7/site-packages/PIL', '/anaconda/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg']
version: 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]

However, when I run anaconda's python in the terminal, here is what I get

Alexs-MacBook-Pro:lib alex$ /anaconda/bin/python2.7
Python 2.7.7 |Anaconda 1.9.1 (x86_64)| (default, Jun  2 2014, 12:48:16) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org

And the result with default's python

Alexs-MacBook-Pro:lib alex$ /usr/bin/python
Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

So it seems that, even if I'm specifying another path for the executable, OSX's default python is called (and conflicts with anaconda's libraries that I'm trying to import later).

My question is thus simple: what am I doing wrong and why do the paths that I specify through Py_Setxxx do not point to the right executable ?

Thank you very much for your help!

Alex

Answer 1

In my case, using Py_SetPythonHome() causes conflicts and I am not able to import the libraries from my virtual environment. What I did was to remove Py_SetPythonHome() and point the PYTHONPATH environment variable to the site-packages inside my virtual environment before calling Py_Initialize(). In short:

QString python_path = "PATH_TO_ENVIRONMENT/lib/python3.11/site-packages";
setenv("PYTHONPATH", python_path.toStdString().c_str(), 1);

Py_Initialize();
if (!Py_IsInitialized())
{
  qDebug() << "Failed to initialize Python interpreter.";
  return;
}
PyRun_SimpleString("import numpy"); // import modules from my venv
Py_Finalize();

Answer 2

The answer Tomas provided helped me. I added a few other options on a similar post:

https://stackoverflow.com/a/46922332/8828614

There was a partial answer in the post you linked.

Option 1: Run your program as follows

LD_LIBRARY_PATH=/path_to_anaconda/lib ./program

Option 2: Run the following command in the terminal, then run your program

export LD_LIBRARY_PATH=/path_to_anaconda/lib ./program

Option 3: Add the following line to the end of your .bashrc file

LD_LIBRARY_PATH=/path_to_anaconda/lib

Why do you have to do this when embedding python, but not when running the interpreter normally? I have no idea, but if some Python/C wizard stumbles on this post I'd love to know why.

Answer 3

Use "import os" and "print(os.sys.path)" to get python home, then:

Py_SetPythonHome((wchar_t*)L"/home/c/anaconda3/lib/python35.zip:"
        "/home/c/anaconda3/lib/python3.5:"
        "/home/c/anaconda3/lib/python3.5/plat-linux:"
        "/home/c/anaconda3/lib/python3.5/lib-dynload:"
        "/home/c/anaconda3/lib/python3.5/site-packages:"
        "/home/c/anaconda3/lib/python3.5/site-packages/setuptools-27.2.0-py3.5.egg:"
        "/home/c/anaconda3/lib/python3.5/site-packages/Sphinx-1.4.6-py3.5.egg");

Answer 4

I've experienced the same problem. The solution for me was to call program like this:

DYLD_LIBRARY_PATH=/path_to_anaconda/lib ./program

It's because the shared libraries used at runtime were loaded from wrong, original OSX directory. (LD_LIBRARY_PATH in linux)