gtest assertions in non-test code

Question

I'm working on a C++ library, and I'm using gtest for unit testing. I want to add ASSERT_* statements to the library code itself, not just the unit test code. I want these ASSERTions to cause a unit test to fail if the code is run under a unit test, or turn into regular asserts if the code is not running under a unite test.

Something like:

if(gtest::is_running)
    ASSERT_TRUE(...);
else
    assert(...);

How can I achieve that?

Answer 1

I've basically use these set of sources. Their are pretty standalone.

To make it works you have to do these steps:

1. Use unit tests compiled from the library sources instead of linkage with the library file (this is easy thing to do with cmake).
2. Extract unit tests and benchmark tests into standalone project with the UNIT_TESTS definition has been defined for unit tests and has not for benchmark tests.
3. include the utility/assert.hpp somethere in a main header of both unit tests and main project BEFORE the gtest/gtest.hpp header inclusion.
4. use ASSERT_TRUE/ASSERT_EQ/etc instead of assert macro

NOTE: In case of benchmark tests you should not define the UNIT_TESTS definition, otherwise the assert definitions does slow down the execution.

utility/assert.hpp

UPD1

• fixed an assert expression evaluation, do evaluate it only once

'

#pragma once

#include "debug.hpp"

#ifdef UNIT_TESTS
#include <gtest/gtest.h>
#endif

#include <cassert>

#define ASSERT_FAIL_BREAK_ON_ATTACHED_DEBUGGER(exp, precondition) \
    if (!(precondition)); else if(!!(exp)); else ::utility::debug_break()

#ifdef GTEST_FAIL

#ifdef _MSC_VER
    #if _MSC_VER < 1600 // < MSVC++ 10 (Visual Studio 2010)
        #error lambda is not supported
    #endif
#else
    #if __cplusplus < 201103L
        #error lambda is not supported
    #endif
#endif

// TIPS:
//  * all lambdas captured by reference because of the error in the MSVC 2015:
//    `error C3493 : '...' cannot be implicitly captured because no default capture mode has been specified`
//  * if debugger is attached but `::testing::GTEST_FLAG(break_on_failure)` has not been setted,
//    then an assertion does a post break.

// gtest asserts rebind with the `void` error workaround (C++11 and higher is required)
#undef ASSERT_TRUE
#define ASSERT_TRUE(condition) [&]() -> void { \
        const bool is_success = ::utility::is_true(condition); \
        const bool break_on_failure = ::testing::GTEST_FLAG(break_on_failure); \
        if (break_on_failure) { \
            GTEST_TEST_BOOLEAN_(is_success, #condition, false, true, GTEST_FATAL_FAILURE_); \
        } else { \
            GTEST_TEST_BOOLEAN_(is_success, #condition, false, true, GTEST_NONFATAL_FAILURE_); \
        } \
        ASSERT_FAIL_BREAK_ON_ATTACHED_DEBUGGER(is_success, !break_on_failure); \
    }()
#undef ASSERT_FALSE
#define ASSERT_FALSE(condition) [&]() -> void { \
        const bool is_success = ::utility::is_false(condition); \
        const bool break_on_failure = ::testing::GTEST_FLAG(break_on_failure); \
        if (break_on_failure) { \
            GTEST_TEST_BOOLEAN_(is_success, #condition, true, false, GTEST_FATAL_FAILURE_); \
        } else { \
            GTEST_TEST_BOOLEAN_(is_success, #condition, true, false, GTEST_NONFATAL_FAILURE_); \
        } \
        ASSERT_FAIL_BREAK_ON_ATTACHED_DEBUGGER(is_success, !break_on_failure); \
    }()

#if !GTEST_DONT_DEFINE_ASSERT_EQ
#undef ASSERT_EQ
#define ASSERT_EQ(val1, val2) [&]() -> void { \
        const ::testing::AssertionResult exp_value = ::testing::internal::EqHelper<GTEST_IS_NULL_LITERAL_(val1)>::Compare(#val1, #val2, val1, val2); \
        const bool break_on_failure = ::testing::GTEST_FLAG(break_on_failure); \
        if (break_on_failure) { \
            GTEST_ASSERT_(exp_value, GTEST_FATAL_FAILURE_); \
        } else { \
            GTEST_ASSERT_(exp_value, GTEST_NONFATAL_FAILURE_); \
        } \
        ASSERT_FAIL_BREAK_ON_ATTACHED_DEBUGGER(exp_value, !break_on_failure); \
    }()
#endif

#if !GTEST_DONT_DEFINE_ASSERT_NE
#undef ASSERT_NE
#define ASSERT_NE(val1, val2) [&]() -> void { \
        const ::testing::AssertionResult exp_value = ::testing::internal::CmpHelperNE(#val1, #val2, val1, val2); \
        const bool break_on_failure = ::testing::GTEST_FLAG(break_on_failure); \
        if (break_on_failure) { \
            GTEST_ASSERT_(exp_value, GTEST_FATAL_FAILURE_); \
        } else { \
            GTEST_ASSERT_(exp_value, GTEST_NONFATAL_FAILURE_); \
        } \
        ASSERT_FAIL_BREAK_ON_ATTACHED_DEBUGGER(exp_value, !break_on_failure); \
    }()
#endif

#if !GTEST_DONT_DEFINE_ASSERT_LE
#undef ASSERT_LE
#define ASSERT_LE(val1, val2) [&]() -> void { \
        const ::testing::AssertionResult exp_value = ::testing::internal::CmpHelperLE(#val1, #val2, val1, val2); \
        const bool break_on_failure = ::testing::GTEST_FLAG(break_on_failure); \
        if (break_on_failure) { \
            GTEST_ASSERT_(exp_value, GTEST_FATAL_FAILURE_); \
        } else { \
            GTEST_ASSERT_(exp_value, GTEST_NONFATAL_FAILURE_); \
        } \
        ASSERT_FAIL_BREAK_ON_ATTACHED_DEBUGGER(exp_value, !break_on_failure); \
    }()
#endif

#if !GTEST_DONT_DEFINE_ASSERT_LT
#undef ASSERT_LT
#define ASSERT_LT(val1, val2) [&]() -> void { \
        const ::testing::AssertionResult exp_value = ::testing::internal::CmpHelperLT(#val1, #val2, val1, val2); \
        const bool break_on_failure = ::testing::GTEST_FLAG(break_on_failure); \
        if (break_on_failure) { \
            GTEST_ASSERT_(exp_value, GTEST_FATAL_FAILURE_); \
        } else { \
            GTEST_ASSERT_(exp_value, GTEST_NONFATAL_FAILURE_); \
        } \
        ASSERT_FAIL_BREAK_ON_ATTACHED_DEBUGGER(exp_value, !break_on_failure); \
    }()
#endif

#if !GTEST_DONT_DEFINE_ASSERT_GE
#undef ASSERT_GE
#define ASSERT_GE(val1, val2) [&]() -> void { \
        const ::testing::AssertionResult exp_value = ::testing::internal::CmpHelperGE(#val1, #val2, val1, val2); \
        const bool break_on_failure = ::testing::GTEST_FLAG(break_on_failure); \
        if (break_on_failure) { \
            GTEST_ASSERT_(exp_value, GTEST_FATAL_FAILURE_); \
        } else { \
            GTEST_ASSERT_(exp_value, GTEST_NONFATAL_FAILURE_); \
        } \
        ASSERT_FAIL_BREAK_ON_ATTACHED_DEBUGGER(exp_value, !break_on_failure); \
    }()
#endif

#if !GTEST_DONT_DEFINE_ASSERT_GT
#undef ASSERT_GT
#define ASSERT_GT(val1, val2) [&]() -> void { \
        const ::testing::AssertionResult exp_value = ::testing::internal::CmpHelperGT(#val1, #val2, val1, val2); \
        const bool break_on_failure = ::testing::GTEST_FLAG(break_on_failure); \
        if (break_on_failure) { \
            GTEST_ASSERT_(exp_value, GTEST_FATAL_FAILURE_); \
        } else { \
            GTEST_ASSERT_(exp_value, GTEST_NONFATAL_FAILURE_); \
        } \
        ASSERT_FAIL_BREAK_ON_ATTACHED_DEBUGGER(exp_value, !break_on_failure); \
    }()
#endif

#define ASSERT(x) ASSERT_TRUE(x)

#else

#ifndef ASSERT_IMPL
#define ASSERT_IMPL(exp) assert(exp)
#endif

#ifdef _DEBUG

#define ASSERT_TRUE(exp) ASSERT_IMPL(exp)
#define ASSERT_FALSE(exp) ASSERT_IMPL(!(exp))

#define ASSERT_EQ(v1, v2) ASSERT_IMPL((v1) == (v2))
#define ASSERT_NE(v1, v2) ASSERT_IMPL((v1) != (v2)))
#define ASSERT_LE(v1, v2) ASSERT_IMPL((v1) <= (v2))
#define ASSERT_LT(v1, v2) ASSERT_IMPL((v1) < (v2))
#define ASSERT_GE(v1, v2) ASSERT_IMPL((v1) >= (v2))
#define ASSERT_GT(v1, v2) ASSERT_IMPL((v1) > (v2))

#define ASSERT(exp) ASSERT_IMPL(exp)

#else

#define ASSERT_TRUE(exp) (::utility::is_true(exp), (void)0)
#define ASSERT_FALSE(exp) (::utility::is_false(exp), (void)0))

#define ASSERT_EQ(v1, v2) (::utility::is_equal(v1, v2), (void)0)
#define ASSERT_NE(v1, v2) (::utility::is_not_equal(v1, v2), (void)0)
#define ASSERT_LE(v1, v2) (::utility::is_less_or_equal(v1, v2), (void)0)
#define ASSERT_LT(v1, v2) (::utility::is_less(v1, v2), (void)0)
#define ASSERT_GE(v1, v2) (::utility::is_greater_or_equal(v1, v2), (void)0)
#define ASSERT_GT(v1, v2) (::utility::is_greater(v1, v2), (void)0)

#define ASSERT(exp) ::utility::is_true(exp)

#endif

#endif

namespace utility
{
    // TIPS:
    // * to capture parameters by reference in macro definitions for single evaluation
    // * to suppress `unused variable` warnings like: `warning C4101: '...': unreferenced local variable`
    template<typename T>
    inline bool is_true(const T & v)
    {
        return !!v; // to avoid warnings of truncation to bool
    }

    template<typename T>
    inline bool is_false(const T & v)
    {
        return !v; // to avoid warnings of truncation to bool
    }

    template<typename T1, typename T2>
    inline bool is_equal(const T1 & v1, const T2 & v2)
    {
        return v1 == v2;
    }

    template<typename T1, typename T2>
    inline bool is_not_equal(const T1 & v1, const T2 & v2)
    {
        return v1 != v2;
    }

    template<typename T1, typename T2>
    inline bool is_less_or_equal(const T1 & v1, const T2 & v2)
    {
        return v1 <= v2;
    }

    template<typename T1, typename T2>
    inline bool is_less(const T1 & v1, const T2 & v2)
    {
        return v1 < v2;
    }

    template<typename T1, typename T2>
    inline bool is_greater_or_equal(const T1 & v1, const T2 & v2)
    {
        return v1 >= v2;
    }

    template<typename T1, typename T2>
    inline bool is_greater(const T1 & v1, const T2 & v2)
    {
        return v1 > v2;
    }
}

utility/debug.hpp

#pragma once

namespace utility
{
    void debug_break(bool breakCondition = true);
    bool is_under_debugger();
}

utility/debug.cpp

#include "debug.hpp"
#include "platform.hpp"

#if defined(UTILITY_PLATFORM_WINDOWS)
#include <windows.h>
#include <intrin.h>
#elif defined(UTILITY_PLATFORM_POSIX)
#include <sys/ptrace.h>
#include <signal.h>
static void signal_handler(int) { }
#else
#error is_under_debugger is not supported for this platform
#endif


namespace utility {

void debug_break(bool breakCondition)
{
    // avoid signal if not under debugger
    if (breakCondition && is_under_debugger()) {
#if defined(UTILITY_COMPILER_CXX_MSC)
        __debugbreak(); // won't require debug symbols to show the call stack, when the DebugBreak() will require system debug symbols to show the call stack correctly
#elif defined(UTILITY_PLATFORM_POSIX)
        signal(SIGTRAP, signal_handler);
#else
#error debug_break is not supported for this platform
#endif
    }
}

bool is_under_debugger()
{
#if defined(UTILITY_PLATFORM_WINDOWS)
    return !!::IsDebuggerPresent();
#elif defined(UTILITY_PLATFORM_POSIX)
    return ptrace(PTRACE_TRACEME, 0, NULL, 0) == -1;
#endif
}

}

utility/platform.hpp

#pragma once

// linux, also other platforms (Hurd etc) that use GLIBC, should these really have their own config headers though?
#if defined(linux) || defined(__linux) || defined(__linux__) || defined(__GNU__) || defined(__GLIBC__)
#  define UTILITY_PLATFORM_LINUX
#  define UTILITY_PLATFORM_POSIX
#  if defined(__mcbc__)
#     define UTILITY_PLATFORM_MCBC
#     define UTILITY_PLATFORM_SHORT_NAME "MCBC"
#  elif defined( __astra_linux__ )
#     define UTILITY_PLATFORM_ASTRA_LINUX
#     define UTILITY_PLATFORM_SHORT_NAME "Astra Linux"
#  else
#     define UTILITY_PLATFORM_SHORT_NAME "Linux"
#  endif
#elif defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__) || defined(__DragonFly__) // BSD:
#  define UTILITY_PLATFORM_BSD
#  define UTILITY_PLATFORM_POSIX
#  define UTILITY_PLATFORM_SHORT_NAME "BSD"
#elif defined(sun) || defined(__sun) // solaris:
#  define UTILITY_PLATFORM_SOLARIS
#  define UTILITY_PLATFORM_POSIX
#  define UTILITY_PLATFORM_SHORT_NAME "Solaris"
#elif defined(__CYGWIN__) // cygwin is not win32:
#  define UTILITY_PLATFORM_CYGWIN
#  define UTILITY_PLATFORM_POSIX
#  define UTILITY_PLATFORM_SHORT_NAME "Cygwin"
#elif defined(_WIN32) || defined(__WIN32__) || defined(WIN32) // win32:
#  define UTILITY_PLATFORM_WINDOWS
#  define UTILITY_PLATFORM_SHORT_NAME "Windows"
#  if defined(__MINGW32__)  //  Get the information about the MinGW runtime, i.e. __MINGW32_*VERSION.
#     include <_mingw.h>
#  endif
#elif defined(macintosh) || defined(__APPLE__) || defined(__APPLE_CC__) // MacOS
#  define UTILITY_PLATFORM_APPLE
#  define UTILITY_PLATFORM_POSIX
#  define UTILITY_PLATFORM_SHORT_NAME "MacOS"
#elif defined(__QNXNTO__)  // QNX:
#  define UTILITY_PLATFORM_QNIX
#  define UTILITY_PLATFORM_POSIX
#  define UTILITY_PLATFORM_SHORT_NAME "QNX"
#elif defined(unix) || defined(__unix) || defined(_XOPEN_SOURCE) || defined(_POSIX_SOURCE)
#  define UTILITY_PLATFORM_UNIX
#  define UTILITY_PLATFORM_POSIX
#  define UTILITY_PLATFORM_SHORT_NAME "Unix"
#else
#   error Unknown platform
#endif

#if defined(__GNUC__)
#   define UTILITY_COMPILER_CXX_GCC
#   define UTILITY_COMPILER_CXX "gcc"
#   define UTILITY_COMPILER_CXX_VERSION __GNUC__
#   if __GNUC__ < 4
#     error "Unsuported gcc version"
#   endif
#elif defined(_MSC_VER)
#   define UTILITY_COMPILER_CXX_MSC
#   define UTILITY_COMPILER_CXX "MS VisualC"
#   define UTILITY_COMPILER_CXX_VERSION _MSC_VER
#else
#   error "Unknown compiler"
#endif

Answer 2

Here is what I ended up doing, following @Josh Kelley's advice:

I've switched from assert to BOOST_ASSERT. Instead of including boost/assert.hpp I've added my own assert.hpp file that includes the Boost file, defines BOOST_ENABLE_ASSERT_HANDLER and a BOOST_ASSERT_HANDLER function pointer (to the exact same type as the Boost assert handler).

I've also included my own Boost assert handler (::boost::assertion_failed) that outputs the assertion information to std::cerr and calls the function pointed to by BOOST_ASSERT_HANDLER if one exists. If there isn't one, it just assert(false)s.

In my test main, I point BOOST_ASSERT_HANDLER to a function that simply calls EXPECT_FALSE(true).

And that's it. Now I can have ordinary asserts when not running under gtest, and gtest-integrated asserts when running under gtest.

Answer 3

You could use preprocessor directives.

When compiling with gtest, tell your compiler to define something like "GTEST_ON", then in your code:

#ifdef GTEST_ON
    ASSERT_TRUE(...);
#else
    assert(...);
#endif

Answer 4

What about approaching this from the alternate direction? Instead of changing your gtest behavior, change your assert's behavior.

Boost.Assert, for example, provides a BOOST_ASSERT macro that, by default, behaves identically to assert. However, if BOOST_ENABLE_ASSERT_HANDLER is defined, then it instead looks for a ::boost::assertion_failed function, which you must provide. You could design your library code to build with standard assertion behavior outside of the test suite and with a ::boost::assertion_failed that calls gtest's FAIL() inside of a test suite.

If you don't want to use Boost, it would be trivial to implement something similar yourself.

This would require building your library twice (once for the test suite, once for regular use), which may not fit well with your overall goals.

Answer 5

Even if this were technically possible (I don't think it is), I really don't believe that making your production code depend on the test framework is a good idea.

The main reasons are robustness, separation of concerns, and decoupling: introducing test-specific conditionals in production code makes the code unnecessarily harder to understand, and may reduce the trustworthiness of your test suite (after all, your tests won't stress the exact same paths your production code will go through).

Also, one day you may want to change something in the testing environment (e.g. the version of the unit test framework, or the unit test framework itself), and this dependency might force you to modify the production code accordingly, at the risk of introducing new bugs.

If what you want to verify is that your assertions actually fire when the client violates the function's preconditions (i.e. if you want to test that preconditions are correctly validated by your assertions), then this proposal may be relevant to you as well as the library which inspired it, Bloomberg's BDE.

If this is not a viable technology for your project, perhaps you may consider adopting a strategy based on Dependency Inversion. The simplest possible approach is to:

1. Define an abstract class Verifier with an abstract member function verify() taking a bool;
2. Derive an AssertingVerifier class from it (to be used in production code) that overrides verify() and forwards its argument to assert(). Both Verifier and AssertVerifier would live in your production code;
3. In your unit test project, define a second derived class, GracefulTestVerifier, that overrides verify() and forwards its argument to ASSERT_TRUE() - or by doing whatever you think is most appropriate;
4. Figure out the best way of injecting a Verifier into your production code - several possibilities exist, but telling which one fits best requires detailed knowledge of your design. You would then inject an AssertVerifier in a regular execution environment, and a GracefulTestVerifier in a testing environment.

This way, execution may flow from the production code to the test framework without your production code being physically dependent on the test framework itself.