Control degree of parallelism with std::async

Question

Is there a way to explicitly set/limit the degree of parallelism (= the number of separate threads) used by std::async and related classes?

Perusing the thread support library hasn’t turned up anything promising.

As close as I could figure out, std::async implementations (usually?) use a thread pool internally. Is there are standardised API to control this?

For background: I’m in a setting (shared cluster) where I have to manually limit the number of cores used. If I fail to do this, the load sharing scheduler throws a fit and I’m penalised. In particular, std::thread::hardware_concurrency() holds no useful information, since the number of physical cores is irrelevant for the constraints I’m under.

Here’s a relevant piece of code (which, in C++17 with parallelism TS, would probably be written using parallel std::transform):

auto read_data(std::string const&) -> std::string;

auto multi_read_data(std::vector<std::string> const& filenames, int ncores = 2) -> std::vector<std::string> {
    auto futures = std::vector<std::future<std::string>>{};

    // Haha, I wish.
    std::thread_pool::set_max_parallelism(ncores);

    for (auto const& filename : filenames) {
        futures.push_back(std::async(std::launch::async, read_data, filename));
    }

    auto ret = std::vector<std::string>(filenames.size());
    std::transform(futures.begin(), futures.end(), ret.begin(),
            [](std::future<std::string>& f) {return f.get();});
    return ret;
}

From a design point of view I’d have expected the std::execution::parallel_policy class (from parallelism TS) to allow specifying that (in fact, this is how I did it in the framework I designed for my master thesis). But this doesn’t seem to be the case.

Ideally I’d like a solution for C++11 but if there’s one for later versions I would still like to know about it (though I can’t use it).

Answer 1

As other people have noted, std::async doesn't let you do this.

Yet (but see the May 2022 update at the end of the answer)

You're describing one of the simpler use-cases of Executors which are currently still bouncing around the design-space of C++ Standardisation, specifically right now in Study Group 1: Concurrency.

Since reading WG21 standards proposals can be a slog, they authors have helpfully linked to both a prototype header-only reference implementation and some example code.

It even includes a static thread pool, and an example of almost exactly what you want: async_1.cpp

#include <experimental/thread_pool>
#include <iostream>
#include <tuple>

namespace execution = std::experimental::execution;
using std::experimental::static_thread_pool;

template <class Executor, class Function>
auto async(Executor ex, Function f)
{
  return execution::require(ex, execution::twoway).twoway_execute(std::move(f));
}

int main()
{
  static_thread_pool pool{1};
  auto f = async(pool.executor(), []{ return 42; });
  std::cout << "result is " << f.get() << "\n";
}

---

Thank you to @jared-hoberock for pointing me at P0668R0 as the much simpler followup to P0443R1 which I had referenced in an earlier version of this answer.

This simplification has been applied, and now there's both a paper describing the rationale (P0761R0), and a much simpler version of the standard wording in P0443R2.

---

As of July 2017, the only actual guess I've seen on delivery of this is: Michael Wong, editor of the Concurrency TS --- the standardisation vehicle for Executors --- feels "confident that it will make it into C++20".

---

I'm still getting Stack Overflow Points™ for this answer, so here's a May 2022 update:

Executors didn't land in C++20.

"A Unified Executors Proposal for C++" reached revision 14 (P0443R14) in 2020, and a new paper std::execution (P2300R5) is proposed as a follow-on; See sections 1.8 and 1.9 for the reasons for the new paper and differences from P0443.

Notably:

A specific thread pool implementation is omitted, as per LEWG direction.

The "Do in a thread-pool" example from std::execution looks like:

using namespace std::execution;

scheduler auto sch = thread_pool.scheduler();

sender auto begin = schedule(sch);
sender auto hi = then(begin, []{
    std::cout << "Hello world! Have an int.";
    return 13;
});
sender auto add_42 = then(hi, [](int arg) { return arg + 42; });

auto [i] = this_thread::sync_wait(add_42).value();

There's a lot to process here. And the last decade of work has pretty much abandoned "std::async and related classes", so perhaps the actual answer to this question is no longer

Yet

but

No, and there never will be. There'll be a different model where you can do that instead.

c.f. the Motivation section of P2300R5

std::async/std::future/std::promise, C++11’s intended exposure for asynchrony, is inefficient, hard to use correctly, and severely lacking in genericity, making it unusable in many contexts.

P2453R0 records the rough feelings of the LEWG participants around this current approach, and also how it interacts with the existing Networking TS, i.e. Asio, which has its own concurrency model. My reading of the polls and comments says neither is likely to land in C++23.

---

I'm still getting Stack Overflow Points™ for this answer, so here's a June 2023 update:

Executors didn't land in C++23.

P2300 looks to be getting some traction, it's up to P2300R7, and there's both an experimental reference implementation and further standardisation exploration based on it, e.g., P2882R0 and P2500R1+P2690R1. That latter link suggests P2300 is targeted for C++26, which was also LEWG's weak consensus last year, albeit with half the participants and fewer comments.

Answer 2

No. std::async is opaque, and you have no control over it's usage of threads, thread pools or anything else. As a matter of fact, you do not even have any guarantee that it would use a thread at all - it might as well execute in the same thread (potentially, note @T.C. comment below), and such implementation would still be conformant.

C++ threading library was never supposed to handle fine-tuning of OS / hardware specifics of threads management, so I am afraid, in your case you will have to code for the proper support yourself, potentially using OS-provided thread control primitives.