CODEWITHSUNDEEP
pythonc++eigenpybind11
fabian
fabian

Reputation: 1881

Can I use pybind11 to pass a numpy array to a function accepting a Eigen::Tensor?

Can I use pybind1 to pass a three-dimensional numpy array to a c++ function accepting an Eigen::Tensor as argument. For example, consider the following c++ function: 

Eigen::Tensor<double, 3> addition_tensor(Eigen::Tensor<double, 3> a,
                                         Eigen::Tensor<double, 3> b) {
    return a + b;
}

after compiling the function, importing it to python and passing a the numpy array np.ones((1, 2, 2)) to it, I receive the following error message: 

TypeError: addition_tensor(): incompatible function arguments. The following argument types are supported:
    1. (arg0: Eigen::Tensor<double, 3, 0, long>, arg1: Eigen::Tensor<double, 3, 0, long>) -> Eigen::Tensor<double, 3, 0, long>

I am in particular surprised about not being able to pass a three dimensional numpy array as I can pass a two dimensional numpy array to a function accepting an Eigen::MatrixXd, as: 

Eigen::MatrixXd addition(Eigen::MatrixXd a, Eigen::MatrixXd b) { return a + b; }

The entire code I used for this example is:

#include <eigen-git-mirror/Eigen/Dense>
#include <eigen-git-mirror/unsupported/Eigen/CXX11/Tensor>
#include "pybind11/include/pybind11/eigen.h"
#include "pybind11/include/pybind11/pybind11.h"

Eigen::MatrixXd addition(Eigen::MatrixXd a, Eigen::MatrixXd b) { return a + b; }

Eigen::Tensor<double, 3> addition_tensor(Eigen::Tensor<double, 3> a,
                                         Eigen::Tensor<double, 3> b) {
    return a + b;
}

PYBIND11_MODULE(example, m) {
    m.def("addition", &addition, "A function which adds two numbers");
    m.def("addition_tensor", &addition_tensor,
          "A function which adds two numbers");
}

I compiled the code above with g++ -shared -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`. Does somebody have an idea how I can a three-dimensional numpy array to a function accepting a three-dimensional Eigen::Tensor?

Upvotes: 5

Views: 3771

Answers (2)

fabian
fabian

Reputation: 1881

Thanks for @John Zwinck's answer, I could achieve what I was looking for. In case somebody is interested, here is the replication: 

#include <eigen-git-mirror/Eigen/Dense>
#include <eigen-git-mirror/unsupported/Eigen/CXX11/Tensor>
#include "pybind11/include/pybind11/eigen.h"
#include "pybind11/include/pybind11/numpy.h"
#include "pybind11/include/pybind11/pybind11.h"

Eigen::Tensor<double, 3, Eigen::RowMajor> getTensor(
    pybind11::array_t<double> inArray) {
    // request a buffer descriptor from Python
    pybind11::buffer_info buffer_info = inArray.request();

    // extract data an shape of input array
    double *data = static_cast<double *>(buffer_info.ptr);
    std::vector<ssize_t> shape = buffer_info.shape;

    // wrap ndarray in Eigen::Map:
    // the second template argument is the rank of the tensor and has to be
    // known at compile time
    Eigen::TensorMap<Eigen::Tensor<double, 3, Eigen::RowMajor>> in_tensor(
        data, shape[0], shape[1], shape[2]);
    return in_tensor;
}

pybind11::array_t<double> return_array(
    Eigen::Tensor<double, 3, Eigen::RowMajor> inp) {
    std::vector<ssize_t> shape(3);
    shape[0] = inp.dimension(0);
    shape[1] = inp.dimension(1);
    shape[2] = inp.dimension(2);
    return pybind11::array_t<double>(
        shape,  // shape
        {shape[1] * shape[2] * sizeof(double), shape[2] * sizeof(double),
         sizeof(double)},  // strides
        inp.data());       // data pointer
}

pybind11::array_t<double> addition(pybind11::array_t<double> a,
                                   pybind11::array_t<double> b) {
    Eigen::Tensor<double, 3, Eigen::RowMajor> a_t = getTensor(a);
    Eigen::Tensor<double, 3, Eigen::RowMajor> b_t = getTensor(b);
    Eigen::Tensor<double, 3, Eigen::RowMajor> res = a_t + b_t;
    return return_array(res);
}

PYBIND11_MODULE(example, m) {
    m.def("addition", &addition, "A function which adds two numbers");
}

In contrast to the suggestion in the link John referred to, I didn't mind using RowMajor storage order for Eigen::Tensor. I saw this storage order being used several times in the tensorflow code too. I do not know if the code above unnecessarily copies data though.

Upvotes: 3

John Zwinck
John Zwinck

Reputation: 249502

It is not directly supported, here's some discussion (including some code to do the mapping if you want to add that to your project): https://github.com/pybind/pybind11/issues/1377

Upvotes: 5

Related Questions