Overhead of std::function vs virtual function call for type erasure

Question

Say I have a templated class that wraps its template argument to provide some extra functionality, like the ability to persist the object's state to disk:

template<typename T>
class Persistent {
    std::unique_ptr<T> wrapped_obj;
public:
    Persistent(std::unique_ptr<T> obj_to_wrap);
    void take_snapshot(int version);
    void save(int to_version);
    void load(int to_version);
}

I want to have another class, let's call it PersistentManager, store a list of these templated Persistent objects and call their member methods without knowing their template parameters. There are two ways I can see to do that: use std::function to erase the template type from each method, or use an abstract base class and virtual function calls.

Using std::function, each Persistent object would be capable of returning a bundle of std::functions bound to its members:

struct PersistentAPI {
    std::function<void(int)> take_snapshot;
    std::function<void(int)> save;
    std::function<void(int)> load;
}

template<typename T>
PersistentAPI Persistent<T>::make_api() {
    using namespace std::placeholders;
    return {std::bind(&Persistent<T>::take_snapshot, this, _1),
            std::bind(&Persistent<T>::save, this, _1),
            std::bind(&Persistent<T>::load, this, _1)}
}

Then the PersistentManager can store a list of PersistentAPIs, and have a method like this:

void PersistentManager::save_all(int version) {
    for(PersistentAPI& bundle : persistents) {
        bundle.save(version);
    }
}

Using inheritance, I would create an abstract class with no template parameters that defines each of Persistent's methods as virtual, and make Persistent inherit from it. Then the PersistentManager can store pointers to this base class, and call the Persistent methods through virtual function calls:

class AbstractPersistent {
public:
    virtual void take_snapshot(int version) = 0;
    virtual void save(int to_version) = 0;
    virtual void load(int to_version) = 0;
}
template<typename T>
class Persistent : public AbstractPersistent {
...
}

void PersistentManager::save_all(int version) {
    for(AbstractPersistent* obj : persistents) {
        obj->save(version);
    }
}

Both of these approaches add some overhead to the function call from PersistentManager: rather than dispatching the function call directly to a Persistent instance, they require going through an intermediary layer, either the std::function object or the virtual function table in AbstractPersistent.

My question is, which approach adds less overhead? Since these are both fairly opaque parts of the standard library, I don't have a good sense of how "expensive" a std::function call is compared to a virtual function call through a base class pointer.

(I've found a few other questions on this site asking about the overhead of std::function, but they all lack a specific alternative to compare against.)

Answer 1

I was hesitating a bit to answer this question, since it could easily boil down to opinions. I have been using std::function in a project, so I might just as well share my two cents (and you can decide what to do with the input).

Firstly, I would like to iterate what's already been said in the comments. If you actually want to see the performance, you have to do some benchmarking. Only after benchmarking, you can derive your conclusions.

Luckily, you can use quick-bench for quick benchmarking(!). I fed the benchmark with your two versions, adding a state that is increased for each call, and a getter for the variable:

// Type erasure:
struct PersistentAPI {
    std::function<void(int)> take_snapshot;
    std::function<void(int)> save;
    std::function<void(int)> load;
    std::function<int()> get;
};

// Virtual base class
class AbstractPersistent {
public:
    virtual void take_snapshot(int version) = 0;
    virtual void save(int to_version) = 0;
    virtual void load(int to_version) = 0;
    virtual int get() = 0;
};

Each function simply increases an integer in the corresponding class, and returns it with get() (hoping that the compiler does not remove all unnecessary code).

The result is in favor of virtual functions, and for both Clang and GCC, we have around 1.7 speed difference (https://quick-bench.com/q/wUbPp8OdtzLZv8H1VylyuDnd2pU, you can change compiler and recheck).

Now to the analysis: why is the abstract class seemingly quicker? Well, there are more indirections with std::function, but also there's another indirection in the wrapping before, when we call std::bind(!). Listening to Scott Meyers, lambdas are to prefer over std::bind, not only for their ease of syntax for people (std::placeholders is no beauty), but their of syntax for the compiler! A lambda call is easier to inline.

Inlining is very important for performance. If a explicit call can be avoided by added the code where we call, we can save some cycles!

Changing std::bind to lambdas, and performing again, we have very similar performance between std::function and inheritance (for both Clang and GCC): https://quick-bench.com/q/HypCbzz5UMo1aHtRpRbrc9B8v44.

So, why are they similar? For Clang and GCC, std::function is internally using inheritance. Type erasure, as it is implemented here, is simply hiding the polymorphism.

(Note that this benchmark might be misleading, since the call for both cases could be completely inlined, thus no indirection is used at all. The test case might have to be a bit more tricky to trick the compiler.)

So let's say you have either Clang and GCC as compilers, which method should you use?

The PersistentAPI is more flexible, since actually take_snapshot, save and load are basically function pointers, and do not need to be assigned to a single class! With

struct PersistentAPI {
    std::function<void(int)> take_snapshot;
    std::function<void(int)> save;
    std::function<void(int)> load;
};

, it is fully reasonable as a developer to believe that PersistentAPI is meant to dispatch to multiple objects, and not just a * single one*. take_snapshot could for example dispatch to a free function, whereas save and load to two different classes. Is this the flexibility you want? Then that's what you should use. Generally, I would use std::function through the API to let the user register a callback to any callable of choice.

If you want to use type erasure, but want to hide the inheritance for some reason, you could build your own version. std::function accepts all types having operator(), we can build one that accepts all classes having the interface "take_snapshot, save and load". It's good to practice!

// probably there is a better name for this class
class PersistentTypeErased {
public:
    template<typename T>
    PersistentTypeErased(T t) : t_(std::make_unique<Model<T>>(t)) {}

    void take_snapshot(int version) { t_->take_snapshot(version); }
    void save(int to_version) { t_->save(to_version); }
    void load(int to_version) { t_->load(to_version); }
private:
    struct Concept
    {
        virtual void take_snapshot(int version) = 0;
        virtual void save(int to_version) = 0;
        virtual void load(int to_version) = 0;
    };
    template<typename T>
    struct Model : Concept
    {
        Model(T t) : t_(t) {}
        void take_snapshot(int version) { t_.take_snapshot(version); }
        void save(int to_version) { t_.save(to_version); }
        void load(int to_version) { t_.load(to_version); }
        T t_;
    };
    std::unique_ptr<Concept> t_;
};

The technique is similar to std::function, and now you probably also can see how type erasure uses polymorphism under the hood. You can see how it is used here.