std::thread access to a function loaded from a shared library

Question

On Ubuntu, I have a shared library mylibrary.so, with a function AlphaFunction. I want to load this function in C++ using dlopen, and then call it in two different threads. However, this is giving me run-time errors, presumably because the two threads are both trying to access the same memory where the function is stored.

The library itself controls a robot arm via USB, and the actual run-time error I get is: LIBUSB_ERROR_NO_DEVICE returned by the Write operation.

I know how to use std::atomic for dealing with shared variables, but what about a shared function?

For example:

void Foo(int (*FooFunction)())
{
    while(true)
    {
        FooFunction();
    }
}

void Bar(int (*BarFunction)())
{
    while(true)
    {
        BarFunction();
    }
}


int main()
{
    void* api_handle = dlopen("mylibrary.so", RTLD_NOW|RTLD_GLOBAL);
    int (*MoveRobot)() = (int (*)()) dlsym(api_handle, "Move");

    std::thread t1(Foo, MoveRobot);
    std::thread t2(Bar, MoveRobot);

    t1.join();
    t2.join();

    return 0;
}

Answer 1

I've had a look at the comments. Here's a solution that covers all concerns:

• the robot library is not thread safe, and
• all calls to the robot library must be on the same thread

This answer proposes a solution in which a third thread is started up which acts as the robot request marshaller. The other threads post tasks to this thread's queue, which are executed one at a time, with the result of the call being returned via a future on which the caller can wait.

#include <thread>
#include <mutex>
#include <queue>
#include <future>
#include <functional>

// these definitions here just to make the example compile
#define RTLD_NOW 1
#define RTLD_GLOBAL 2
extern "C" void* dlopen(const char*, int);
extern "C" void* dlsym(void*, const char*);


struct RobotCaller final
{
    RobotCaller()
    {
        _library_handle = dlopen("mylibrary.so", RTLD_NOW|RTLD_GLOBAL);
        _Move = (int (*)()) dlsym(_library_handle, "Move");

        // caution - thread starts. do not derive from this class
        start();
    }

    void start()
    {
        _robot_thread = std::thread([this]{
            consume_queue();
        });
    }

    ~RobotCaller() {
        if (_robot_thread.joinable()) {
            std::unique_lock<std::mutex> lock(_queue_mutex);
            _should_quit = true;
            lock.unlock();
            _queue_condition.notify_all();
            _robot_thread.join();
        }

        // close library code goes here
    }

    std::future<int> Move()
    {
        return queue_task(_Move);
    }

private:
    void consume_queue() {
        ;
        for(std::unique_lock<std::mutex> lock(_queue_mutex) ; !_should_quit ; lock.lock()) {
            _queue_condition.wait(lock, [this]{
                return _should_quit || (!_task_queue.empty());
            });

            if (!_task_queue.empty()) {
                auto task = std::move(_task_queue.front());
                _task_queue.pop();
                lock.unlock();
                task();
            }
        }
    }

    std::future<int> queue_task(int (*f)())
    {
        std::packaged_task<int()> task(f);
        auto fut = task.get_future();
        std::unique_lock<std::mutex> lock(_queue_mutex);
        _task_queue.push(std::move(task));
        return fut;
    }

private:
    // library management
    void* _library_handle = nullptr;
    int (*_Move)() = nullptr;

    // queue management
    std::thread _robot_thread;
    std::queue<std::packaged_task<int()>> _task_queue;
    bool _should_quit = false;
    std::mutex _queue_mutex;
    std::condition_variable _queue_condition;
};

void Foo(std::function<std::future<int>()> FooFunction)
{
    while(true)
    {
        // marshal the call onto the robot queue and wait for a result
        auto result = FooFunction().get();
    }
}

void Bar(std::function<std::future<int>()> BarFunction)
{
    while(true)
    {
        // marshal the call onto the robot queue and wait for a result
        auto result = BarFunction().get();
    }
}


int main()
{
    RobotCaller robot_caller;

    std::thread t1(Foo, std::bind(&RobotCaller::Move, &robot_caller));
    std::thread t2(Bar, std::bind(&RobotCaller::Move, &robot_caller));

    t1.join();
    t2.join();

    return 0;
}