CODEWITHSUNDEEP
c++runtimetemplate-meta-programmingvirtual-functionslow-latency
bedman3
bedman3

Reputation: 1

When is virtual dispatch faster than function templates in C++ runtime?

Most knows that template meta programming is in general faster than virtual dispatch in C++ due to types of templates were decided in compile time while virtual functions required runtime lookup on vtable. In runtime context, what are the scenarios when virtual dispatch function is faster than compiled template function?

Upvotes: 0

Views: 125

Answers (2)

Ahmed AEK
Ahmed AEK

Reputation: 18090

Templates are almost always faster than virtual functions, because virtual dispatch has indirection, while templated code can have no indirection. but you can construct a case where the opposite is true and virtual functions would be faster.

On architectures based on Von Neumann (all PCs and smartphones) Instructions are also Data, that needs to be loaded from the RAM to the CPU instruction caches to be executed, and badly written templated code that instantiated too many unnecessary templates can produce a lot of instructions.

If the code cannot fit into caches, and the memory bus was tied down loading actual data, the templated code execution will be slow, because the computer will be running into memory bandwidth issues. while virtual dispatch (type-erasure) produces less code, so you end up with less memory bandwidth usage and therefore faster code.

In those cases there are possible optimizations to alleviate this issue, like splitting templated classes to a non-templated base and a templated derived type, or try to increase the compiler's ICF (identical code folding).

there is a talk that recently demonstrated this effect Save Time, Space & a Little Sanity With std::function_ref - David Ledger, the benchmarks are shown at the end where function_ref is faster than templated code. virtual dispatch is slightly slower than function_ref but it can have the same scaling behavior because they are both type-erasure. and you need thousands of template instantiations to have any notable difference.

For normal code, you don't need to worry about either effect. Use whatever fits your requirements. templates for compile time polymorphism and virtual functions for runtime polymorphism. but beware of the disadvantages of both, and have considerable limits to the use of them. Don't use templated code where you don't need to, and don't use virtual functions where you don't need to, the problem is not using templates, the problem is instantiating too many unnecessary templates, also all applications can work well with a few virtual functions, but its performance will surely plumet if all functions are virtual. (staring at Java).

Upvotes: 1

Aleksandr Medvedev
Aleksandr Medvedev

Reputation: 9008

Long story short - dynamic dispatch is never faster than a static dispatch, because the latter doesn't involve any runtime overhead. I.e. by the time your program starts, it already knows all addresses of statically dispatched functions and it doesn't need to calculate it when calling.

One of C++ mottos is "You don't pay for what you don't use" and dynamic dispatch is actually the price you pay when you need to support a virtual function call in your class.

Upvotes: 0

Related Questions