ANTLR4: grun errors while using separate grammar (ClassCastException)

Question

Description

I'm trying to create a custom language that I want to separate lexer rules from parser rules. Besides, I aim to divide lexer and parser rules into specific files further (e.g., common lexer rules, and keyword rules).

But I don't seem to be able to get it to work.

Although I'm not getting any errors while generating the parser (.java files), grun fails with Exception in thread "main" java.lang.ClassCastException.

Note

I'm running ANTLR4.7.2 on Windows7 targeting Java.

Code

I created a set of files that closely mimic what I intend to achieve. The example below defines a language called MyLang and separates lexer and parser grammar. Also, I'm splitting lexer rules into four files:

1. // MyLang.g4
   parser grammar MyLang;
   
   options { tokenVocab = MyLangL; }
   
   prog
       : ( func )* END
       ;
   
   func
       : DIR ID L_BRKT  (stat)* R_BRKT
       ;
   
   stat
       : expr SEMICOLON
       | ID OP_ASSIGN expr SEMICOLON
       | SEMICOLON
       ;
   
   expr
       : expr OPERATOR expr
       | NUMBER
       | ID
       | L_PAREN expr R_PAREN
       ;
   

2. // MyLangL.g4
   lexer grammar MyLangL;
   
   import SkipWhitespaceL, CommonL, KeywordL;
   
   @header {
   package com.invensense.wiggler.lexer;
   }
   
   @lexer::members {   // place this class member only in lexer
   Map<String,Integer> keywords = new HashMap<String,Integer>() {{
       put("for",          MyLangL.KW_FOR);
       /* add more keywords here */
   }};
   }
   
   ID  :   [a-zA-Z]+
           {
           if ( keywords.containsKey(getText()) ) {
               setType(keywords.get(getText())); // reset token type
           }
           }
       ;
   
   DIR
       : 'in'
       | 'out'
       ;
   
   END : 'end' ;
   

3. // KeywordL.g4
   lexer grammar KeywordL;
   
   @lexer::header {    // place this header action only in lexer, not the parser
   import java.util.*;
   }
   
   // explicitly define keyword token types to avoid implicit def warnings
   tokens {
       KW_FOR
       /* add more keywords here */
   }
   

4. // CommonL.g4
   lexer grammar CommonL;
   
   NUMBER
       : FLOAT
       | INT
       | UINT
       ;
   
   FLOAT
       : NEG? DIGIT+ '.' DIGIT+ EXP?
       | INT
       ;
   
   INT
       : NEG? UINT+
       ;
   
   UINT
       : DIGIT+ EXP?
       ;
   
   OPERATOR
       : OP_ASSIGN
       | OP_ADD
       | OP_SUB
       ;
   
   OP_ASSIGN   : ':=';
   OP_ADD      : POS;
   OP_SUB      : NEG;
   
   L_BRKT          : '[' ;
   R_BRKT          : ']' ;
   L_PAREN         : '(' ;
   R_PAREN         : ')' ;
   SEMICOLON       : ';' ;
   
   fragment EXP
       : [Ee] SIGN? DIGIT+
       ;
   
   fragment SIGN
       : POS
       | NEG
       ;
   
   fragment POS: '+' ;
   fragment NEG : '-' ;
   fragment DIGIT : [0-9];
   

5. // SkipWhitespaceL.g4
   lexer grammar SkipWhitespaceL;
   
   WS
       :   [ \t\r\n]+ -> channel(HIDDEN)
       ;
   

Output

Here is the exact output I receive from the code above:

ussjc-dd9vkc2 | C:\M\w\s\a\l\example
§ antlr4.bat .\MyLangL.g4

ussjc-dd9vkc2 | C:\M\w\s\a\l\example
§ antlr4.bat .\MyLang.g4

ussjc-dd9vkc2 | C:\M\w\s\a\l\example
§ javac *.java

ussjc-dd9vkc2 | C:\M\w\s\a\l\example
§ grun MyLang prog -tree
Exception in thread "main" java.lang.ClassCastException: class MyLang
        at java.lang.Class.asSubclass(Unknown Source)
        at org.antlr.v4.gui.TestRig.process(TestRig.java:135)
        at org.antlr.v4.gui.TestRig.main(TestRig.java:119)

ussjc-dd9vkc2 | C:\M\w\s\a\l\example
§

Answer 1

ANTLR4.13 SEPARATE LEXER AND PARSER GRAMMAR HOWTO

Setup

Let foo be the name of the grammar. in this example foo = Hello.

• parser grammar must be named fooParser, in file fooParser.g4
• lexer grammar must be named fooLexer, in file fooLexer.g4

Then each must be processed by antlr4, first lexer then parser:

• antlr4 fooLexer.g4 fooParser.g4

Then compile:

• javac *.java

Then grun:

• grun foo -gui etc.

EXAMPLE

Assume the following:

• grammar name = Hello
• parserfile = HelloParser.g4 (contents shown below)
• lexerfile = HelloLexer.g4 (contents shown below)

The sequence of commands needed:

1. antlr4 HelloLexer.g4 HelloParser.g4
2. javac *.java
3. grun Hello r -gui -tokens
4. hello parrt
5. control-d

Results of grun are now displayed.

HelloParser.g4 contents

parser grammar HelloParser;
options { tokenVocab=HelloLexer;}
r: HELLO ID;

HelloLexer.g4 contents

lexer grammar HelloLexer;
HELLO: 'hello';
ID: [a-z]+;
WS: [ \t\r\n]+ ->skip;

Answer 2

Rename both of your file with MyLangParser and MyLangLexer, then run grun MyLang prog -tree