CODEWITHSUNDEEP
javatreeantlrtokengrammar
Val
Val

Reputation: 762

ANTLR language - Tree not created (EOF - syntax error)

I have apparently a mistake in the following language definition :

grammar Hello;


object  :
ALL* NAME ALL* '{' 
    (ALL* | (ALL* NAME ALL* NAME)* | (ALL* object)*)*
'}' ALL*;

ALL     :
(~('{' | '}' | '"'))+ -> skip;      // All but braces and double quotes

NAME    :
'"' ALL* '"';


When I'm trying to read a file or add content directly (with run.bat Hello object -gui), the parser give me a syntax error :mismatched input '<EOF>' expecting NAME. And I just have a tree with a single node : 'object'.


Here is the Java source :

public static void main(String[] args) throws Exception {
    HelloLexer lexer = new HelloLexer(new ANTLRFileStream("gamemodes.txt"));
    TokenStream tokenStream = new CommonTokenStream(lexer);
    HelloParser parser = new HelloParser(tokenStream);

    System.out.println(lexer.getAllTokens().size());

    parser.setBuildParseTree(true);

    // Tree Creation
    RuleContext tree = parser.object();
    tree.inspect(parser);
}


And finally, the file structure (with some spaces/characters anywhere (without braces and double quotes)) :

...
"objName"
{
    ...
    "innerObjName1"
    {
        "firstProperty"  "firstResult"
        ...
        "secondProp"     ""
    }

    "innerObjName2"
    {
        "firstProperty"  "firstResult"
        "secondProp"     ""
    }
}
...


NOTE : I'm on Windows.

Thanks !

Upvotes: 0

Views: 661

Answers (2)

Sam Harwell
Sam Harwell

Reputation: 99889

reportAttemptingFullContext is not an error. It's simply letting you know that ANTLR 4 is using the section of its internal implementation that uses the complete parse context to ensure accuracy. The message is included because the full context algorithm is a bit slower than the SLL algorithm it attempts first.

Edit: If you are using a non-standard distribution, you may need to specify the following option to make sure the parse tree is built. This option is true by default in the official distribution.

parser.setBuildParseTree(true);

Edit 2: Covering grammar issues.

Here is the object rule from the original grammar with only formatting changes.

object // intermediate form 1
    :   ALL* NAME ALL* '{' 
        (   ALL*
        |   (ALL* NAME ALL* NAME)*
        |   (ALL* object)*
        )*
        '}' ALL*
    ;

This rule is highly ambiguous. As a quick example, note that the following modified grammar will actually match the same input.

object // intermediate form 2
    :   ALL* NAME ALL* '{' 
        (   ALL
        |   ALL* NAME ALL* NAME
        |   ALL* object
        )*
        '}' ALL*
    ;

Another simplification removes the Unnecessary ALL* prefixes from 2 alts.

object // final form
    :   ALL* NAME ALL* '{' 
        (   ALL
        |   NAME ALL* NAME
        |   object
        )*
        '}' ALL*
    ;

As a final change, I would create an entry rule that ends with an explicit EOF:

entry : object EOF;

To parse your input, call entry() instead of object() to ensure that all of your input gets parsed. If you need the ObjectContext object, it is available by calling EntryContext.object().

Upvotes: 2

Bart Kiers
Bart Kiers

Reputation: 170178

When you skip a lexer rule, you must not use it in your parser rules.

Try something like this:

grammar Hello;

parse
 : object EOF
 ;

object
 : NAME (OBRACE object+ CBRACE | NAME)
 ;

NAME   : '"' ~["{}]* '"';
OBRACE : '{';
CBRACE : '}';
OTHER  : ~["{}]+ -> skip;

Upvotes: 1

Related Questions