How to achieve function unit testing during compilation?

Question

I'm writing a function which applies a complicated formula to a bunch of integers. When I make lots of changes, I always compile then do a bunch of test cases at run-time to make sure I didn't break anything, so I thought "Okay, I'll just put some test cases in my code".

But of course this is undesirable because the function takes a while to do its thing and the tests are run every time I execute the program. Is there any simple way to automate this so that some cases are checked at compile-time instead? Here is a small example which shows my current (not-so-desirable) method.

#include <iostream>

int binom(int, int);

int main() {
  //I don't want these tests to happen at runtime...
  bool test1 = (binom(5,2)!=10); //5 choose 2 should be 10
  bool test2 = (binom(7,4)!=35); //7 choose 4 should be 35
  if (test1 || test2) {
    std::cout << "Your algorithm has an error!";
    return 0;
  }

  int n, k;
  std::cout << "n: "; std::cin >> n;
  std::cout << "k: "; std::cin >> k;
  std::cout << n << " choose "
            << k << " is " << binom(n,k);
  return 0;
}

int binom(int n, int k) {
  if (k > n/2)
    k = n-k;
  int nCk = 1, i=1;
  for ( ; i<=k; ++i) {
    nCk *= n-k+i;
    nCk /= i;
  }
  return nCk;
}

(The function I'm writing is more complicated than n choose k, I just wanted to have a simple example of what I'm trying to do.) I've read about unit tests and something called "Compile time function execution" but I didn't think the former was applicable and I didn't really understand the latter, so I'm wondering if there's a simple way to understand those or achieve something similar.

Answer 1

You can use constexpr to evaluate binom at compile-time and static_assert to test it. Note that you will also need to express iteration in terms of tail recursion. In the case of binom example, this is straightforward:

#include <iostream>

constexpr int binom_loop(int n, int k, int i, int nCk) {
  return i <= k
    ? binom_loop(n, k, i + 1, nCk * (n-k+i) / i)
    : nCk;
}

constexpr int binom(int n, int k) {
  return binom_loop(n, k > n/2 ? n-k : k, 1, 1);
}

// assert that the calculation works - at compile-time
static_assert (binom(5, 2) == 10,
               "5 choose 2 should be 10");

int main() {
  // enum demonstrates that binom() can be evaluated at compile-time
  // if called with literal args
  enum {
    X = binom(5, 2),
  };
  std::cout << int(X) << '\n';   // prints 10
}

Answer 2

Generally, I would suggest you think of compiling and running a separate unit test (in this case the unit being a function) as two mandatory steps in releasing your program.

Starting with the status-quo of having the tests run on startup of the application. Unless you specifically want your program to do a self-check every time it runs, you want to separate your tests from the running of the program. You can make a post-build step that runs the tests every time that you compile your code. If the tests pass, it will publish the tested (sometimes called promoted) executable to an output folder. That way you always know that your released executables have been well tested.

Cases where you would want to run your tests every time you start the application is if there is something about the operating environment that the program needs to test - such as make sure that the correct 3rd party resources / instances / services are available and supported. Even then, it is better to cover as much as you can in your build-time testing first so that you avoid unexpected surprises when your program goes live.

Regarding "Compile time function execution", it is more a way to set a constant value to some output of a function for well-known arguments. Having said that, there are some new c++11 features (if your compiler supports them) that will allow you to check a const expression at runtime (such as static_assert) see user4815162342 answer for an example. Personally, I would have some reservations about using this for tests because I would feel that the testing and code were too coupled and that I was violating the principle of 'separation of concerns'. Specifically: -

• I would worry about bloated compile times.
• I might sometimes want to run my program even if one of the tests are failing. Commenting the tests out would run the risk of me forgetting not to put them back in.
• This would only work with specific functions adhering to quite strict constraints, so it would only be possible with certain types of test methods.
• I might want to run a subset of the tests without recompiling or commenting out tests.
• I might want to run a different set of tests on the exact same executable without the risk of introducing change by recompiling.

These are similar arguments to why tests should not be inline with code (see here and here as a start).

On the other hand, I can see some value for very simple, fundamental algorithmic or mathematical functions that can be evaluated very quickly such as the one that you have provided. It is nice that the compiler can check this for you and you have that good feeling that any executable has passed those fundamental tests. It is a bit of a philosophical quandary. If it suits your case, then I hope that you have the C++11 features to use them.

Answer 3

One would typically split their test cases out from their main program. You're going to have a hard time testing functions declared and defined in your main file however. Optimally these functions/types would be declared in their own header and defined in their own implementation file.

You can then write a second program which uses the same headers and executes the tests. So..

// binomial.h
#ifndef BINOMIAL_H
#define BINOMIAL_H

int binom(int, int);

#endif

// binomial.cpp

#include "binomial.h"

int binom(int n, int k) {
  if (k > n/2)
    k = n-k;
  int nCk = 1, i=1;
  for ( ; i<=k; ++i) {
    nCk *= n-k+i;
    nCk /= i;
  }
  return nCk;
}

Now, you create a new project, with it's own main.hpp.

// unit test - main.cpp

#include "binomial.h"
#include <assert>

int main() {
  bool test1 = binom(5,2) == 10; //5 choose 2 should be 10
  bool test2 = binom(7,4) == 35; //7 choose 4 should be 35
  assert(test1 && test2);
  return 0;
}

Now you run that every time you build and you know that your function is not broken (well, these tests could be far more comprehensive, but you get the idea). There also exist libraries dedicated to unit testing which can make your life easier.