How to hide C# unit tests?

Question

For quick experiments, I prefer to use "unit tests" –i.e. annotating one (or more) method with [TestMethod]– over creating a tiny project with a Main method.

A test method has some advantages over a tiny project:

• it is quicker to write a method and annotate it with [TestMethod] than to create a new project;
• the test method can use facilities provides by the visual studio test tools such as test fixtures shared with similar experiments (via [TestInitialize]);
• the method can be executed by right clicking and selecting "Run test" (in visual studio 2015); and,
• using Assert() instead of inspecting a value via the debugger or via Console.WriteLine() automatically documents the intent and the results of the experiment and makes it repeatable without manual intervention.

A problem is that these experiments show up in the visual studio test explorer, this causes clutter and makes it difficult to distinguish real unit tests from experiments. (I'm not using any other testing reporting tools, CI, etc., but I assume that the problem would occur there as well.)

Is there a way to have the advantages of using [TestMethod] for experiments and avoid the clutter that it causes at the same time?

I tried the following to solve this problem:

• Have the [TestClass] and the [TestMethod] attributes as comments and uncomment them when you want to run the experiment.

  This is what I currently do. It works, but running an experiment is somewhat cumbersome.

• Use [Ignore] attribute.

  Clutters the test explorer view with ignored tests and requires commenting out the [Ignore] attribute to run the test.

• Put experiments in different project; as suggested by Veverke or annotate the test with a [TestCategory] attribute as suggested by C Sharper.

  The visual studio test explorer can group tests by either class, duration, outcome, traits, project. To avoid the clutter: Choosing traits allows to keep the tree of tests annotated with [TestCategory("Experiments")] closed or choosing projects allows keeping the "experiments.proj" tree closed.

  When using the grouping to distinguish experiments from tests, the other groupings in the test explorer cannot be used.

TL;DR

How to write experiments (using visual studio unit tests) without the experiments cluttering the reports about real unit tests?

Answer 1

Combining the answers:

1. Do you REALLY want to abuse unit tests as experiments? ^{(comment by Jamie Rees and in answer by C Sharper)}

   Perhaps Linqpad ([https://www.linqpad.net/]) or something similar is better for experiments.

2. If you want to abuse unit tests, mark each experiment method with a [TestCategory]. This allows grouping the experiments in the test explorer and selecting which tests to run.^{(From answer by C Sharper)}

3. Use a build configuration to define a conditional compilation symbol, for example EXPERIMENTS. Use #if EXPERIMENTS to enable or disable the experiments. Use a solution build configuration to be able to enable or disable the experiments from the toolbar.(From answer by Kasper van den Berg

Example:

#if EXPERIMENTS
[TestClass]
public class MyExperiment1
{
    [TestMethod, TestCategory("Experiment")]
    public void Method1()
    {
        …
    }
}
#endif

Answer 2

Experiments can be disabled via an ifdef, e.g.:

#if EXPERIMENTS
[TestClass]
public class MyExperiment1
{
    [TestMethod]
    public void Method1()
    {
        …
    }
}
#endif

To enable the experiment add #define EXPERMIMENTS as first line the file; however, this is still cumbersome. The conditional compilation symbol (EXPERIMENTS) can also be defined via project properties. To improve on this, create an build configuration and a solution configuration and define the conditional compilation symbols only for the build configuration.

Answer 3

Basically you misuse Unit Tests for something for which it was not intended. But the only solution that comes in my mind is to use [TestCategory] attribute to control when a test runs and in which list it should be.

[TestMethod(), TestCategory("Nightly"), TestCategory("Weekly"), TestCategory("ShoppingCart")]
public void DebitTest()
{

}