Unit-testing the semantic analysis visitors of an ANTLR-based tree

Question

I'm developing a small compiler. In past compilers I've worked with I just ignored unit-testing, doing all my testing via system-tests. It worked reasonably well, but I felt it always to be far from perfect. This time I'm tempted to try to make things in a different way.

Focusing now specifically on the semantic analysis phase, I'd like to have each one of my visitors having a set of associated unit-tests. But to be able to do so, I'll have to accomplish one of two things. I either:

1. Structure my grammar in such a way as allowing me to not have to define the package, define the class, etc, when I just want to unit-test a method-related visitor. An example would be testing that there are no duplicated variables defined in a given method -- ideally I don't need to know this method is inside a class that is inside a package. Following this path, my ANTLR grammar would be able to parse things such as void m() { int x = 1; int y = 2; int x = 3; } although in the real world (for the end user) this wouldn't be an allowed source code, as in my language a method must always be contained in a class that is always contained in a package;

2. Take the time to learn the ins and outs of ANTLR and how to programatically create nodes in my junit-tests. Something such as

   MethodNode method = new MethodNode(); method.setName("m"); method.addChildren(new VarDecl("x", new IntegerExpr("1"))); method.addChildren(new VarDecl("y", new IntegerExpr("2"))); method.addChildren(new VarDecl("x", new IntegerExpr("3")));

   This may be a suitable solution if ANTLR's API is "friendly" for this kind of ends. I've looked up a tiny bit and got scared by some gory details in the nodes constructors (ProgramContext(ParserRuleContext parent, int invokingState) <- this definitely doesn't seem to have been made to be used by me, the programmer, but I may be wrong..);

3. Create my own AST and as first step in the parsing phase I convert ANTLR's tree to my tree. On one hand this would be kinda cumbersome, on the other hand, having total control over the tree could facilitate other things down the road.

I can only assume this is a recurrent concern among compiler developers. How do you handle this specific situation?

Thanks

Answer 1

I did have the same problem (at least I hope so, I admit I did not quite understand what you meant with knowing about packages and classes in the package) and since I did not find another solution I created an example myself - the full source can be found on GitHub.

The most important part of the project is obviously the Visitor:

public class MyVisitor extends DemoBaseVisitor<String> {
    @Override
    public String visitPlus(final DemoParser.PlusContext ctx) {
        return visit(ctx.left) + " PLUS " + visit(ctx.right);
    }

    @Override
    public String visitLiteralNumber(final DemoParser.LiteralNumberContext ctx) {
        return ctx.getText();
    }
}

Any my test for that visitor:

public class MyVisitorTest {
    private final MyVisitor myVisitor = new MyVisitor();

    @Test
    public void visitPlus_joinsOperatorsWithWordPLUSAsSeparator() throws Exception {
        // setup
        final DemoParser.PlusContext plusNode = mock(DemoParser.PlusContext.class);
        plusNode.left = mockForVisitorResult(DemoParser.ExpressionContext.class, "2");
        plusNode.right = mockForVisitorResult(DemoParser.ExpressionContext.class, "4");

        // execution
        final String actual = myVisitor.visitPlus(plusNode);

        // evaluation
        assertEquals(actual, "2 PLUS 4");
    }

    private<T extends RuleContext> T mockForVisitorResult(final Class<T> nodeType, final String visitResult) {
        final T mock = mock(nodeType);
        when(mock.accept(myVisitor)).thenReturn(visitResult);
        return mock;
    }

    @Test
    public void visitLiteralNumber_returnsTextValueOfNumber() throws Exception {
        // setup
        final DemoParser.LiteralNumberContext literalNumberNode = mock(DemoParser.LiteralNumberContext.class);
        when(literalNumberNode.getText()).thenReturn("42");

        // execution
        final String actual = myVisitor.visitLiteralNumber(literalNumberNode);

        // evaluation
        assertEquals(actual, "42");
    }
}

This method is similar to you solution number 2:

Take the time to learn the ins and outs of ANTLR and how to programatically create nodes in my junit-tests.

However without learning the ins and outs of ANTLR. Instead IO just use a mocking framework (Mockito) to generate tree nodes as I see fit and also mock the accept method of the tree nodes so that I can mock visiting children without mocking anything on the visitor itself.

Answer 2

A typical solution is creating input files which you expect to behave in a particular way, and then running them through the semantic analyzer to verify that they do in fact behave that way.