How does a C++ compiler choose between deferred and async execution for std::async?

Question

It seems that the default behaviour of std::async heavily favours std::launch::deferred. I'm trying to understand why exactly the default behaviour seemingly never actually spawns asynchronous tasks. Consider this code:

#include <iostream>
#include <future>
#include <thread>
#include <chrono>
#include <vector>


int main(void)
{
    std::vector<std::future<void>> task_list;

    size_t n_tasks = 10; // Let's say this could change at runtime
    // The first two seem interchangeable in this example:
    //auto launch_pol = std::launch::deferred;
    //auto launch_pol = std::launch::async | std::launch::deferred;
    // Only this seems to actually do async tasks:
    auto launch_pol = std::launch::async;

    auto start_time = std::chrono::steady_clock::now();

    // Generate a bunch of tasks
    for (size_t i = 0; i < n_tasks; i++) {
        task_list.emplace_back(std::async(launch_pol,
            [i](){
                std::cout << " Starting task " << i << std::endl;
                // The sleep is emulating, for example, a slow, I/O
                // bound operation
                std::this_thread::sleep_for(std::chrono::seconds(1));
                std::cout << " Stopping task " << i << std::endl;
            }
        ));
        // The following lines are experiments I tried to nudge the
        // task to start doing something.
        if (!task_list.at(i).valid()) {
            std::cout << "Task not valid!" << std::endl;
        }
        task_list.at(i).wait_for(std::chrono::milliseconds(1));
    }


    // Wait for them to complete
    for (auto& task : task_list) {
        task.get();
    }

    std::chrono::duration<double> stop_time =
        std::chrono::steady_clock::now() - start_time;
    std::cout << "Execution time: " << stop_time.count() << std::endl;

    return 0;
}

Notice that I've been experimenting with multiple launch policies. It seems that unless I explicitly state std::launch::async (only!), the compiler will fall back to std::launch::deferred. I tried this with Clang 3.8, gcc 5.4, and this post seems to indicate that MSVC works in the same way.

OK, this is not in contradiction with the C++ standard. I get this is not a bug. If we specify deferred, we may get lazy evaluation, which is (in this case) pretty much the same as a serial execution of my tasks. However, what's the point if the compiler just falls back to std::launch::deferred?

If the compiler is always falling back to lazy evaluation, then calling std::async without std::launch::async seems pointless. I was hoping that the C++ runtime is smart about launching threads (or not) if I use the default launch policy.

Some background: In the problem I'm trying to solve, I'm running a variable number of initialization calls, which are pretty slow, but completely I/O bound (i.e., they wait most of the time for results from elsewhere). The number of these might scale, so I was hoping from some help from the compiler to schedule threads.

Answer 1

Compiler vendors basically all chose to make "pick one" mean "deferred".

This sucks.

They are free to do any logic they choose. They chose to make their logic be "always defer".