CODEWITHSUNDEEP
antlrgrammarantlr4
Andrew Logvinov
Andrew Logvinov

Reputation: 21851

Migration tool for ANTLR grammar

Suppose I have a following simple grammar (query DSL):

grammar TestGrammar;

term : textTerm ;

textTerm : 'Text' '(' T_VALUE '=' STRING+ ')' ;

T_VALUE : 'value' ;
STRING : '"' .+? '"' ;

WS : [ \t\r\n]+ -> skip ;

Then at some point I decide that text term format needs to be changed, for example:

Text(value = "123") -> MyText(val = "123")

How should I approach migrating existing data that users have generated with previous version of grammar?

Upvotes: 2

Views: 185

Answers (1)

quepas
quepas

Reputation: 1003

Assumption

Let's make one simplification of your grammar by introducing token TEXT for 'Text' string.

grammar TestGrammar;

WS : [ \t\r\n]+ -> channel(HIDDEN);  // preserve the whitespaces characters!
T_VALUE : 'value';
STRING : '"' .+? '"';
TEXT : 'Text';

term
    : textTerm;
textTerm
    : TEXT '(' T_VALUE '=' STRING+ ')';

Solution

Now we will utilize the AST listener built by ANTLRv4 tool. This allows us to traverse AST and perform token replacement with TokenStreamRewriter class already mentioned by Lucas Trzesniewski in comments.

import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.TokenStreamRewriter;

public class MigrationTask extends TestGrammarBaseListener {
    private TokenStreamRewriter rewriter;

    public MigrationTask(CommonTokenStream stream) {
        this.rewriter = new TokenStreamRewriter(stream);
    }

    @Override
    public void enterTextTerm(TestGrammarParser.TextTermContext ctx) {
        rewriter.replace(ctx.TEXT().getSymbol(), "MyText");
        rewriter.replace(ctx.T_VALUE().getSymbol(), "val");
    }

    public String getMigrationResult() {
        return rewriter.getText();
    }
}

So, we substitute given token with its replacement whenever we encounter the token during the traversal of AST.

Usage

Now we can execute MigrationTask on given ParseTree and retrive the migration result:

(...)
CommonTokenStream tokens = new CommonTokenStream(lexer);
TestGrammarParser parser = new TestGrammarParser(tokens);
ParseTree tree = parser.term();
ParseTreeWalker walker = new ParseTreeWalker();
MigrationTask migrationTask = new MigrationTask(tokens);
walker.walk(migrationTask, tree);
String result = migrationTask.getMigrationResult(); // Here we retrive migration result !
(...)

Upvotes: 1

Related Questions