Reputation: 1
I have a python library written in c++ and specifically a function that retrieves a value from a redis server using boost async io:
python::object redis::connexion::py_mget(python::object keys) {
int size = python::extract<int>(keys.attr("__len__")())();
std::vector<python::object> objects(size);
std::string error;
// we need to protect the python GIL from multithreaded access
unsigned int todo = size;
std::mutex mutex;
std::condition_variable condition;
std::unique_lock<std::mutex> lock(mutex);
for(auto i : xxs::range(size)) {
const auto& key = python::extract<redis::key>(keys[i])();
(*this)(
[&,i] (std::unique_ptr<xxs::redis::value> value) {
std::unique_lock<std::mutex> lock(mutex);
/* This next line throws segfault */
objects[i] = key.read(std::move(value));
if(!--todo) condition.notify_one();
},
[&] (const std::string& msg) {
std::unique_lock<std::mutex> lock(mutex);
error = msg;
if(!--todo) condition.notify_one();
}
) << redis::get(key.name());
}
while(todo) condition.wait(lock);
if(!error.empty()) THROW(error);
return python::make_list(objects);
}
where key.read calls this function:
virtual python::object read(std::unique_ptr<xxs::redis::value> value) const override {
if(value->is_null()) return python::object();
if(!value->is_blob()) THROW("Invalid value stored in REDIS")
auto pos = value->blob().seekg();
xxs::object obj;
obj.read(pos);
/* This function throws the Segfault */
return python::object(obj);
}
Now, keep in mind:
key.read(std::move(value))
and I used it without a loop, it actually works. However, when I put it in the original context with the async io and the mutexes, then it throws a Segfault (core dumped).The valgrind output is completely unreadable...
Please help me, I'm quite clueless and helpless!
I have tried to use
PyGILState_STATE gstate; gstate = PyGILState_Ensure(); PyGILState_Release(gstate);
in several different places to see if I could improve things. The only thing it managed to was to create deadlocks.
Upvotes: 0
Views: 29