How to strongly factorize code unit testing all parameters of a bunch of functions

Question

I have a bunch of functions, used to customize an algorithm parameters. Functions are getting a different count of parameters of different types (some standard: double or int, other custom classes giving access to double or int values at some point through getters).

All algorithm parameters must be inside valid bounds ([min,max]). I need to write a unit test to make sure the bound checking for every parameter of every function is coded correctly (reach 100% code coverage).

Here is my MCVE:

Code to be tested:

class Object
{
public:
    Object( double value ) : value( value ) {}
    inline const double& getValue() const { return value; }
private:
    double value;
};

static const double minA = 0;
static const double maxA = 100;
static const int minB = 10;
static const int maxB = 20;
static const Object minC = Object( 23.0 );
static const Object maxC = Object( 29.0 );

bool func1( double a )
{
    if ( a < minA )
        return false;
    else if ( a > maxA )
        return false;

    // do something
    return true;
}

bool func2( int b, const Object& c )
{
    if ( b < minB )
        return false;
    else if ( b > maxB )
        return false;
    else if ( c.getValue() < minC.getValue() )
        return false;
    else if ( c.getValue() > maxC.getValue() )
        return false;

    // do something
    return true;
}

Example of test that has to be factorized:

double getValidValue( const std::pair<double,double>& minmax ) { return minmax.first + (minmax.second-minmax.first)/2; }
int getValidValue( const std::pair<int,int>& minmax ) { return minmax.first + (minmax.second-minmax.first)/2; }
Object getValidValue( const std::pair<Object,Object>& minmax ) { return Object( minmax.first.getValue() + (minmax.second.getValue()-minmax.first.getValue())/2); }

double getInvalidLowerValue( const std::pair<double,double>& minmax ) { return minmax.first - 1; }
int getInvalidLowerValue( const std::pair<int,int>& minmax ) { return minmax.first - 1; }
Object getInvalidLowerValue( const std::pair<Object,Object>& minmax ) { return Object( minmax.first.getValue() - 1); }

double getInvalidUpperValue( const std::pair<double,double>& minmax ) { return minmax.second + 1; }
int getInvalidUpperValue( const std::pair<int,int>& minmax ) { return minmax.second + 1; }
Object getInvalidUpperValue( const std::pair<Object,Object>& minmax ) { return Object( minmax.second.getValue() + 1); }

int main ()
{
    // valid cases:
    assert( func1( getValidValue( std::make_pair(minA,maxA) ) ) );
    assert( func2( getValidValue( std::make_pair(minB,maxB) ), getValidValue( std::make_pair(minC,maxC) ) ) );

    // func1 out of bound cases:
    assert( !func1( getInvalidLowerValue( std::make_pair(minA,maxA) ) ) );
    assert( !func1( getInvalidUpperValue( std::make_pair(minA,maxA) ) ) );

    // func2 out of bound cases:
    // two tests won't offer a 100% code coverage!
    //assert( !func2( getInvalidLowerValue( std::make_pair(minB,maxB) ), getInvalidLowerValue( std::make_pair(minC,maxC) ) ) );
    //assert( !func2( getInvalidUpperValue( std::make_pair(minB,maxB) ), getInvalidUpperValue( std::make_pair(minC,maxC) ) ) );

    // func2, first param out of bound cases
    assert( !func2( getInvalidLowerValue( std::make_pair(minB,maxB) ), getValidValue( std::make_pair(minC,maxC) ) ) );
    assert( !func2( getInvalidUpperValue( std::make_pair(minB,maxB) ), getValidValue( std::make_pair(minC,maxC) ) ) );
    // func2, second param out of bound cases
    assert( !func2( getValidValue( std::make_pair(minB,maxB) ), getInvalidLowerValue( std::make_pair(minC,maxC) ) ) );
    assert( !func2( getValidValue( std::make_pair(minB,maxB) ), getInvalidUpperValue( std::make_pair(minC,maxC) ) ) );

    return (0);
}

Note that:

• I use assert to simplify the MCVE (I actually use CPPUnit library and CPPUNIT_ASSERT macro).
• Calling a function with all parameters invalid won't reach 100% code coverage: As func2 checks all parameters in different boolean evaluation, having both parameters invalid will not reach code checking second parameter. A function with n parameters needs 1+2*n calls to be fully tested.
• Modifying the way functions check the input parameters (by having one and only one if statement) to make it easier to reach 100% code coverage is not acceptable (the algorithm is used in a medical device and our goal is to be user every bound of every parameter is tested, no matter how the code is written).

As we have many functions (~20) all with lots of parameters (from 1 to 5), I'd like to ideally end up with a fully factorized solution where the core test code would be:

int main()
{
    testFunc( &func1, /* give parameter bounds min/max for every parameter of func1 */ );
    testFunc( &func2, /* give parameter bounds min/max for every parameter of func2 */ );
}

I tried to use variadic templates here (thought it could help), but I'm not sure this will work and could not find out how to write the testFunc function (specially how to iterate over the arguments and how to have a variable 1+2*n function calls statements, n being the number of arguments...).

Here is what I have so far (not much...and it does not compile), if one wants to use this as a start. But answers with a completely different approach is perfectly acceptable.

template <typename ...Args> void testFunc( bool (*func)( Args ... ), const std::pair<Args...,Args...>& args )
{
     assert( (*func)( /* all getValidValue( args ) ... */ );
     for ( arg : args )
     {
         assert( !(*func)( /* all getValidValue but one getInvalidLowerValue */ );
         assert( !(*func)( /* all getValidValue but one getInvalidUpperValue */ );
     }
}

int main()
{
    testFunc( &func1, std::make_pair( minA, maxA ) );
    testFunc( &func2, std::make_pair( minB, maxB ), std::make_pair( minC, maxC ) );
}

Note: Solutions using boost are acceptable

Answer 1

C++11 implementation could look as follows:

#include <tuple>
#include <utility>
#include <type_traits>
#include <cassert>
#include <iostream>

// CODE TO BE TESTED:

class Object
{
public:
    Object( double value ) : value( value ) {}
    inline const double& getValue() const { return value; }
private:
    double value;
};

static const double minA = 0;
static const double maxA = 100;
static const int minB = 10;
static const int maxB = 20;
static const Object minC = Object( 23.0 );
static const Object maxC = Object( 29.0 );

bool func1( double a )
{
    std::cout << "Calling func1(" << a << ")" << std::endl;
    if ( a < minA )
        return false;
    else if ( a > maxA )
        return false;

    // do something
    return true;
}

bool func2( int b, const Object& c )
{
    std::cout << "Calling func2(" << b << "," << c.getValue() << ")" << std::endl;
    if ( b < minB )
        return false;
    else if ( b > maxB )
        return false;
    else if ( c.getValue() < minC.getValue() )
        return false;
    else if ( c.getValue() > maxC.getValue() )
        return false;

    // do something
    return true;
}

// TESTING CODE:

// integer_sequence implementation

template <class T, T... Vs>
struct integer_sequence { };

template <class T, class, class, class = integer_sequence<T>, class = integer_sequence<T, 0>, class = void>
struct make_integer_sequence_impl;

template <class T, T ICV1, T... Res, T... Pow>
struct make_integer_sequence_impl<T, std::integral_constant<T, ICV1>, std::integral_constant<T, 0>, integer_sequence<T, Res...>, integer_sequence<T, Pow...>, typename std::enable_if<(ICV1 > 0)>::type>: make_integer_sequence_impl<T, std::integral_constant<T, ICV1/2>, std::integral_constant<T, ICV1%2>, integer_sequence<T, Res...>, integer_sequence<T, Pow..., (Pow + sizeof...(Pow))...>> { };

template <class T, T ICV1, T... Res, T... Pow>
struct make_integer_sequence_impl<T, std::integral_constant<T, ICV1>, std::integral_constant<T, 1>, integer_sequence<T, Res...>, integer_sequence<T, Pow...>, void>: make_integer_sequence_impl<T, std::integral_constant<T, ICV1/2>, std::integral_constant<T, ICV1%2>, integer_sequence<T, Pow..., (Res + sizeof...(Pow))...>, integer_sequence<T, Pow..., (Pow + sizeof...(Pow))...>> { };

template <class T, class Res, class Pow>
struct make_integer_sequence_impl<T, std::integral_constant<T, 0>, std::integral_constant<T, 0>, Res, Pow, void> {
   using type = Res;
};

template <class T, T V>
using make_integer_sequence = typename make_integer_sequence_impl<T, std::integral_constant<T, V/2>, std::integral_constant<T, V%2>>::type;

template <size_t V>
using make_index_sequence = make_integer_sequence<size_t, V>;

template <size_t... V>
using index_sequence = integer_sequence<size_t, V...>;

// end of integer_sequence implementation

// helper functions to generate valid/invalid inputs:

// TODO: possibly return values randomly offseted?
double getValidValue( const std::pair<double,double>& minmax ) { return minmax.first + (minmax.second-minmax.first)/2; }
int getValidValue( const std::pair<int,int>& minmax ) { return minmax.first + (minmax.second-minmax.first)/2; }
Object getValidValue( const std::pair<Object,Object>& minmax ) { return Object( minmax.first.getValue() + (minmax.second.getValue()-minmax.first.getValue())/2); }

double getInvalidLowerValue( const std::pair<double,double>& minmax ) { return minmax.first - 1; }
int getInvalidLowerValue( const std::pair<int,int>& minmax ) { return minmax.first - 1; }
Object getInvalidLowerValue( const std::pair<Object,Object>& minmax ) { return Object( minmax.first.getValue() - 1); }

double getInvalidUpperValue( const std::pair<double,double>& minmax ) { return minmax.second + 1; }
int getInvalidUpperValue( const std::pair<int,int>& minmax ) { return minmax.second + 1; }
Object getInvalidUpperValue( const std::pair<Object,Object>& minmax ) { return Object( minmax.second.getValue() + 1); }

// end of helper functions to generate valid/invalid inputs:

template <std::size_t N, std::size_t, class = make_index_sequence<N>>
struct TestFuncImplInnerLoop;

template <std::size_t N, std::size_t J, std::size_t... Is>
struct TestFuncImplInnerLoop<N, J, index_sequence<Is...>> {
    template <class Func, class Tup>
    int operator()(Func func, const std::string& funcName, Tup &tup) {
        std::cout << "Calling " << funcName << " with argument #" << J+1 << " lower than lower bound:" << std::endl;
        assert(!(*func)((J == Is)?getInvalidLowerValue(std::get<Is>(tup)):getValidValue(std::get<Is>(tup))...));
        std::cout << "Calling " << funcName << " with argument #" << J+1 << " greater than upper bound:" << std::endl;
        assert(!(*func)((J == Is)?getInvalidUpperValue(std::get<Is>(tup)):getValidValue(std::get<Is>(tup))...));
        return 0;
    }
};

template <std::size_t N, class = make_index_sequence<N>> 
struct TestFuncImpl;

template <std::size_t N, std::size_t... Is>
struct TestFuncImpl<N, index_sequence<Is...>> {
    template<class Func, class Tup>
    void operator()(Func func, const std::string& funcName, Tup &tup) {
        std::cout << "Calling " << funcName << " with valid arguments:" << std::endl;
        assert((*func)(getValidValue(std::get<Is>(tup))...));
        int falseAsserts[sizeof...(Is)] = { TestFuncImplInnerLoop<N, Is>{}(func, funcName, tup)... };
        (void)falseAsserts;
    }
};

template <class... Args>
void testFunc(bool (*func)(Args...), const std::string& funcName, std::pair<Args, Args>&&... args) {
    auto argsTup = std::make_tuple(args...);
    std::cout << std::endl << "Testing " << funcName << ":" << std::endl;
    TestFuncImpl<sizeof...(Args)>{}(func, funcName, argsTup);
}

// wrapper needed because testFunc can't call function taking const ref as arguments, they need to pass parameters by copy
bool func2Wrapper( int b, Object c )
{
    return func2( b, c );
}

int main() {
    testFunc( &func1, "func1", std::make_pair( minA, maxA ) );
    testFunc( &func2Wrapper, "func2", std::make_pair( minB, maxB ), std::make_pair( minC, maxC ) );
}

This works like a charm (OP edit) and outputs:

Testing func1:
Calling func1 with valid arguments:
Calling func1(50)
Calling func1 with argument #1 lower than lower bound:
Calling func1(-1)
Calling func1 with argument #1 greater than upper bound:
Calling func1(101)

Testing func2:
Calling func2 with valid arguments:
Calling func2(15,26)
Calling func2 with argument #1 lower than lower bound:
Calling func2(9,26)
Calling func2 with argument #1 greater than upper bound:
Calling func2(21,26)
Calling func2 with argument #2 lower than lower bound:
Calling func2(15,22)
Calling func2 with argument #2 greater than upper bound:
Calling func2(15,30)

[live demo]

The code need refactor to apply const reference on Objects cause now it demands use Object by value...