C++ Templates: Are template ínstantiations inlined? Are there drawbacks in performance?

Question

When some C++ entity, such as a structure, class, or function, is declared as a template, then the definitions provided for said entities are solely blue-prints which must be instantiated.

Due to the fact that a template entity must be defined when it is declared (which is commonly header files) I have the conception, which I try to convince myself as wrong, that when after a template has been instantiated it will be inlined by the compiler. I would like to ask if this is so?

The answer for this question raises my suspicion when I read the the paragraph:

"Templates can lead to slower compile-times and possibly larger executable, especially with older compilers."

Slower compile-times are clear as template must be instantiated but why "possibly larger executables"? In what ways should this be interpreted? Should I interpret this as 'many functions are inlined' or 'the executable's size increases if there are many template instantiations, that is the same template is instantiated with a lot of different types, which causes several copies of the same entity to be there'?

In the latter case, does a larger executable size cause the software to run more slowly seeing that more code must be loaded into memory which will in turn cause expensive paging?

Also, as these questions are also somewhat compiler dependent I am interested in the Visual C++ compiler. Generalised answers concerning what most compilers do give a good insight as well.

Thank you in advance.

Answer 1

Due to the fact that a template entity must be defined when it is declared (which is commonly header files)

Not true. You can declare and define template classes, methods and functions separately, just like you can other classes, methods and functions.

I have the conception, which I try to convince myself as wrong, that when after a template has been instantiated it will be inlined by the compiler. I would like to ask if this is so?

Some of it may be, or all of it, or none of it. The compiler will do what it deems is best.

Slower compile-times are clear as template must be instantiated but why "possibly larger executables"? In what ways should this be interpreted?

It could be interpreted a number of ways. In the same way that a bottle of Asprin contains the warning "may induce <insert side-effects here>".

Should I interpret this as 'many functions are inlined' or 'the executable's size increases if there are many template instantiations, that is the same template is instantiated with a lot of different types, which causes several copies of the same entity to be there'?

You won't have several copies of the same entity - the compiler suite is obliged to see to that. Even if methods are inline, the address of the method will:

1. always exist, and
2. be the same address when referenced by multiple compilation units.

What you may find is that you start creating more types than you intended. For example std::vector<int> is a completely different type to std::vector<double>. foo<X>() is a different function to foo<Y>(). The number of types and function definitions in your program can grow quickly.

In the latter case, does a larger executable size cause the software to run more slowly seeing that more code must be loaded into memory which will in turn cause expensive paging?

Excessive paging, probably not. Excessive cache misses, quite possibly. Often, optimising for smaller code is a good strategy for achieving good performance (under certain circumstances such as when there is little data being accessed and it's all in-cache).

Answer 2

Whether they are inlined is always up to the compiler. Non-inlined template instantiations are shared for all similar instantiations.

So a lot of translation units all wanting to make a Foo<int> will share the Foo instantiation. Obviously if Foo<int> is a function, and the compiler decides in each case to inline it then the code is repeated. However the choice to inline is because the optimization of doing so seems highly likely to be superior to a function call.

Technically there could be a corner case where a template causes slower execution. I work on software which has super tight inner loops for which we do a lot of performance measurements. We use a number of template functions and classes and it has yet to show up a degradation compared to writing the code by hand.

I can't be certain but I think you'd have to have a situation where the template: - generated non-inlined code - did so for multiple instantiations - could have been one hand-written function - hand-written function incurs no run-time penalty for being one function (ie: no implicit conversions involving runtime checks)

Then it's possible that you have a case where the single-handwritten-function fits within the CPU's instruction cache but the multiple template instantiations do not.