How to access private members in unit tests?

Question

While developing unit tests, accessing private members to check the internal state of the class can be necessary. Sometimes getter and setter functions are not available, sometimes they are not public.

First method to handle that, writing a preprocessor define writes publis instead of private and protected.

#define protected public
#define private public

Second method is making the test classes friends of the classes.

class test_foo {
};

class foo {
private:
  int mem;

  friend class test_foo;
};

A third method is creating public interface for test.

class foo {

#if defined FOO_UNIT_TEST
public:
  int get_mem() const;
#endif

private:
  int mem;
};

Is there any other method except these methods? Every method has pros and cons. Which one can be thought as best practice?

Thanks.

Answer 1

This is why I feel that testing classes in this detail is fundamentally a bad idea.

It break encapsulation.

Testing works because it is a dispassionate appraisal of the function of a unit from the perspective of the specification, rather than the programming. When writing tests from the perspective of a programmer, you can defeat that purpose.

Who will write the unit test?

If it is the person who wrote the class then he will already have his own idea of how the algorithm works and what values should be where. He will likely write the test to accommodate most of the bugs he introduced to begin with making the test just as buggy as the code and, worse, giving the buggy code a free pass.

If it is someone who did not write the class how will they know what private members should and should not contain? That person will have to examine the internal algorithm in order to discover what values need to be in the private members at what points. They too are likely to be influenced by what the code says rather than what the specification requires. They too may introduce the very bugs that are in the code into the tests giving them a free pass.

The bottom line is that I feel testing classes from the perspective of the implementer is inherently flawed and invites tests that are themselves just as buggy as the code they are testing.

Having a second pair of eyes look over/co-develop the code is a good thing but that's for the coding phase, not the testing.

Tests should be designed from the perspective of the specification and they should be about breaking the contract. They should not be about your ability to spot bugs in an algorithm. That is the programmer's job if the test fails. The tester's job is to focus on cause and effect based on external stimuli. The question they should be asking is "Does the class do what it is advertised as doing?" not "Is the code doing it properly?".

That is not to say the programmer should not code with testing in mind. Classes can be designed to make their contracts more testable. But the person writing the test should not know, or care, how the class does what it does, only that it does it correctly according to specification.

When it comes to testing, specification is king, not implementation.

So I think testing is effective when you are pitting implementation against specification. If you get into looking at the implementation then you start pitting one person's idea of how an implementation should work against another's idea of how an implementation should work and they run the risk of either introduce their own implementation bug into the test or else reinforce the original programmers bug.

Answer 2

There is no "best practice" in this sort of thing. Only different subjective opinions.

Your first option is not a good practice, since redefining language keywords using the preprocessor is specified as giving undefined behaviour. A test case that potentially introduces undefined behaviour could well fail or (even worse) pass for reasons completely unrelated to characteristics of the class under test.

Of the three, I would probably encourage the second approach (a friend class that does unit test). That way, the tests can be implemented in a way that does not rely on any particular features of the class under test, but can check anything it likes.

The third option has the potential disadvantage of clashing with other member functions of the class, or inherited from base classes. I remember one colleague having trouble with a similar technique, when the test function happened to have the same signature as a virtual function declared in a base class (one of those cases of a function named using an english word that had multiple distinct meanings) the test function therefore caused other apparently unrelated tests to fail.

It's probably best to design your classes, as much as possible, so examining the internal state of a class is not necessary. However, I realise there are some V&V regimes where it is necessary to provide separate evidence that a class' member functions are implemented correctly - which can be hard to achieve without examining class private data.

Answer 3

All three solutions are ugly in my opinion. I would say best practice is to not check internal state of your class in unit tests, and not exposing methods just for unit testing. Doing this goes against the principles of TDD.

A good unit test makes no assumptions about internal state and will run without modification if the class implementation is completely changed.

Why is checking the internal state necessary? Maybe you have design issues that are better solved with redesign rather than hacks. Do you have a more thorough example of what you are trying to achieve?