CODEWITHSUNDEEP
c++multithreadingstdcoroutinec++-coroutine
BrainStone
BrainStone

Reputation: 3205

C++ is std::generator thread safe?

I've read through the documentation of std::generator<T> and found no mention of thread-safety.

Now essentially I have a co-routine that generates work elements and I want to consume them with multiple threads. Generating a new work item takes a bit of time, but processing them even longer, hence the co-routine + worker pool approach.

Essentially I'll end up with code like this

auto generator = get_generator();

std::vector<std::thread> threads;

for(int i = 0; i < NUM_THREADS; ++i) {
    threads.emplace_back(worker, generator);
}

And then a worker function without any additional synchronization like this:

void worker(std::generator<...>& generator) {
    for(const auto& element : generator) {
        // processing goes here
    }
}

I'm specifically looking to know if the standard has anything to say about this. And on top if any compiler implementations are thread-safe or not.
After all I'm not writing portable code at the moment, so only some compilers making this thread safe is fine, but for the sake of the internet and future me, knowing what the standard has to say about this would be much more interesting.

Finally if this is not thread safe, how do I make it thread safe the most straightforward way?

Upvotes: 4

Views: 115

Answers (1)

BrainStone
BrainStone

Reputation: 3205

Indeed they are not thread safe. But with a little helper class they can be made thread safe.

I'm sure you can also implement your own generator class that has the synchronization built in, but the following has worked for me just fine:

#include <generator>
#include <mutex>
#include <type_traits>


template <typename T>
class sync_generator {
private:
    using generator_iterator = decltype(std::declval<std::generator<T>>().begin());
    using generator_iterator_sentinel = decltype(std::declval<std::generator<T>>().end());

    std::generator<T> generator;
    generator_iterator it;
    generator_iterator_sentinel end;
    std::mutex mutex;

public:
    // Construct by moving in a generator.
    explicit sync_generator(std::generator<T>&& gen);

    sync_generator(const sync_generator<T>& other) = delete;
    sync_generator(sync_generator<T>&& other) = default;

    sync_generator& operator=(const sync_generator<T>& other) = delete;
    sync_generator& operator=(sync_generator<T>&& other) = default;

    // Returns the next value, or std::nullopt if finished.
    std::optional<T> next();
};

template <typename T>
sync_generator<T>::sync_generator(std::generator<T>&& generator)
    : generator(std::move(generator)), it(this->generator.begin()), end(this->generator.end()) {}

template <typename T>
std::optional<T> sync_generator<T>::next() {
    std::lock_guard<std::mutex> lock(mutex);

    if (it == end) return std::nullopt;

    T value = *it;
    ++it;
    
    return value;
}

Upvotes: 0

Related Questions