CODEWITHSUNDEEP
pythonc++multithreadingopencv
Ulysse Darmet
Ulysse Darmet

Reputation: 41

OpenCV functions lock when called by a Python script itself called by a C++ threaded program

I have a C++ application calling Python functions from several threads. Everything works fine until I try to use an OpenCV function from within Python:

Basically I have two files:

script.py:

import cv2

def foo():
    print('foo_in')
    cv2.imread('sample.jpg')
    print('foo_out')

main.cpp:

#include <pthread.h>
#include <pybind11/embed.h>

pybind11::handle g_main;

void* foo(void*)
{
    g_main.attr("foo")();
}

int main()
{
    pybind11::scoped_interpreter guard;
    pybind11::eval_file("script.py");
    g_main = pybind11::module::import("__main__");

    foo(nullptr);

    pthread_t thread;
    pthread_create(&thread, nullptr, &foo, nullptr);
    pthread_join(thread, nullptr);

    return 0;
}

And when I execute the C++ snippet I get:

foo_in
foo_out
foo_in

...and then it gets stuck forever.

As you can see the first call to cv2.imread returns but not the second one (the one called in an other thread).

When I strace the thread PID I get the following lines:

futex(0x7fe7e6b3e364, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 13961, {1550596187, 546432000}, ffffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7fe7e6b3e3e0, FUTEX_WAKE_PRIVATE, 1) = 0

...printed again and again, which makes me think the thread is waiting for a mutex to be released.

I further tried to understand what was going on by using the backtrace of gdb:

#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x00007fe7e667948f in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#2  0x00007fe7e6679979 in PyEval_RestoreThread () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#3  0x00007fe7e669968b in PyGILState_Ensure () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#4  0x00007fe7e3fa7635 in PyEnsureGIL::PyEnsureGIL (this=<synthetic pointer>) at <opencv>/modules/python/src2/cv2.cpp:83
#5  NumpyAllocator::deallocate (this=<optimized out>, u=0x7fe7a80008c0) at <opencv>/modules/python/src2/cv2.cpp:208
#6  0x00007fe7d88e17c2 in cv::MatAllocator::unmap (this=<optimized out>, u=<optimized out>) at <opencv>/modules/core/src/matrix.cpp:18
#7  0x00007fe7e3fa7dc8 in cv::Mat::release (this=0x7fe7ae8018e0) at <opencv>/modules/core/include/opencv2/core/mat.inl.hpp:808
#8  cv::Mat::~Mat (this=0x7fe7ae8018e0, __in_chrg=<optimized out>) at <opencv>/modules/core/include/opencv2/core/mat.inl.hpp:694
#9  pyopencv_from<cv::Mat> (m=...) at <opencv>/modules/python/src2/cv2.cpp:451
#10 0x00007fe7e3faa08c in pyopencv_cv_imread (args=<optimized out>, kw=<optimized out>) at <opencv>/build/modules/python_bindings_generator/pyopencv_generated_funcs.h:10588
#11 0x00007fe7e6575049 in PyCFunction_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#12 0x00007fe7e66811c5 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#13 0x00007fe7e6711cbc in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#14 0x00007fe7e6711d93 in PyEval_EvalCodeEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#15 0x00007fe7e6599ac8 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#16 0x00007fe7e664e55e in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#17 0x00007fe7e6710947 in PyEval_CallObjectWithKeywords () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#18 0x00000000004369de in pybind11::detail::simple_collector<(pybind11::return_value_policy)1>::call (this=0x7fe7ae801e80, ptr=0x7fe7e6eaef28) at <pybind11>/pybind11/cast.h:1953
#19 0x00000000004334f3 in pybind11::detail::object_api<pybind11::detail::accessor<pybind11::detail::accessor_policies::str_attr> >::operator()<(pybind11::return_value_policy)1> (this=0x7fe7ae801ed0)
    at <pybind11>/pybind11/cast.h:2108
#20 0x0000000000424336 in foo () at main.cpp:11

I tried moving the python interpreter initialization into the foo function and then it worked (I just had to remove the first call to foo as the interpreter can only be initialized once per application).

This makes me think that the cv2.imread function only returns if called in the same thread the interpreter was initialized in.

The same happens if I replace the call to cv2.imread by any other OpenCV function. I tested it on cv2.imwrite and cv2.projectPoints.

Any idea of what is happening and how I can get around it while still being able to call OpenCV functions from different threads?

Upvotes: 2

Views: 980

Answers (1)

Ulysse Darmet
Ulysse Darmet

Reputation: 41

So it turned out the problem was me using Python instructions without holding the GIL (Global Interpreter Lock). The GIL is first held by the thread initializing the interpreter and must be explicitly released before other threads can acquire it.

The reason the execution locked on the cv2.imread instruction and not the print('foo_in') instruction is that the Python interpreter does not ensure it is holding the GIL when called from C++ (which means that any pure Python instruction is executed in a thread-unsafe way). However the C++ code called by the cv2.* instructions under the hood does make sure it is holding the GIL before executing, hence the lock.

I fixed the problem using explicit GIL release and acquisition:

main.cpp

#include <pthread.h>
#include <pybind11/embed.h>

pybind11::handle g_main;

void* foo(void*)
{
    pybind11::gil_scoped_acquire gil;
    g_main.attr("foo")();
}

int main()
{
    pybind11::scoped_interpreter guard;
    pybind11::eval_file("../script.py");
    g_main = pybind11::module::import("__main__");
    pybind11::gil_scoped_release nogil;

    foo(nullptr);

    pthread_t thread;
    pthread_create(&thread, nullptr, &foo, nullptr);
    pthread_join(thread, nullptr);

    return 0;
}

Now everything works fine and I do get the expected output:

foo_in
foo_out
foo_in
foo_out

Upvotes: 2

Related Questions