CODEWITHSUNDEEP
pythonc++multithreadingpybind11
ManuelSchneid3r
ManuelSchneid3r

Reputation: 16121

How to use pybind11 in multithreaded application

I want to run Python in a worker thread. However I get strange segfaults and deadlocks of the threads in the worker pool. How do I correctly use pybind11/Python C API to allow the threads to run the jobs?

I know that it does not make much sense to MT python because of the GIL, but thats an intermediate solution to fit the current architecture until there is a better approach.

Upvotes: 7

Views: 9364

Answers (2)

user3713719
user3713719

Reputation: 325

this works. Wrap the long running c++ code with gil_scoped_release and gil_scoped_acquire

pybind11::gil_scoped_release release;

while (true)
{
    // do something and break
}

pybind11::gil_scoped_acquire acquire;

Upvotes: 6

ryadav
ryadav

Reputation: 514

If you want to do following - From Python you want to run C++ threads to execute different tasks on each thread.

Create a theradpool like https://github.com/progschj/ThreadPool

Write a wrapper class to wrap and bind to Python using PyBind11. Create the instance of ThreadPool and add tasks from Python which in turn creates different threads to execute your tasks.

Disclaimer - I have not tried but this will work :)

Upvotes: 1

Related Questions