ANTLR help - can't parse a normal text but all the other more complex rules work

Question

I have a grammar that is very simple - it parses a bunch of text that was entered by the user for various keywords, and then does string replacements for those keywords while leaving all the other text "as is".

Basically, this grammar should do nothing to most of the text... it should just echo that text... except for when it runs into ${...}, $video{...}, $image{...} and $audio{...}

My grammar mostly works now, but it can't parse normal text like hey using the chunk rule, but for some reason, it parses <p>hey</p> using the chunk rule. Why? How can I get both to use the chunk rule?

Basically, I want CHUNK to catch everything else that is not being picked up by the other rules.

I refactored the grammar a bit to make everything more clear. It would seem that everythingElse is getting confused with label? I don't know though, but hey does not work :(

grammar Text;

@header {
}

@members {
    int numberOfVideos = 0;
    StringBuilder builder = new StringBuilder();

    public String getResult() {
        return builder.toString();
    }
}

text
    :   expression*
    ;

expression
    :   fillInTheBlank 
        {
            builder.append($fillInTheBlank.value);
        }
    |   image 
        {
            builder.append($image.value);
        }
    |   video
        {
            builder.append($video.value);
        }
    |   audio
        {
            builder.append($audio.value);
        }
    |   anchor
        {
            builder.append($anchor.value);
        }
    |   everythingElse
        {
            builder.append($everythingElse.value);
        }
    ;

fillInTheBlank returns [String value]
    :   '${' LABEL '}' 
        {
            $value = "<input type=\"text\" id=\"" +
                $LABEL.text +
                "\" name=\"" + 
                $LABEL.text +
                "\" class=\"FillInTheBlankAnswer\" />";
        }
    ;

image returns [String value]
    :   '$image{' URL '}'
        {
            $value = "<img src=\"" + $URL.text + "\" />";
        }
    ;

video returns [String value]
    :   '$video{' URL '}'
        {
            numberOfVideos++;

            StringBuilder b = new StringBuilder();
            b.append("<div id=\"video1\">Loading the player ...</div>\r\n");
            b.append("<script type=\"text/javascript\">\r\n");
            b.append("\tjwplayer(\"video" + numberOfVideos + "\").setup({\r\n");
            b.append("\t\tflashplayer: \"/trainingdividend/js/jwplayer/player.swf\", file: \"");
            b.append($URL.text);
            b.append("\"\r\n\t});\r\n");
            b.append("</script>\r\n");

            $value = b.toString();
        }
    ;

audio returns [String value]
    :   '$audio{' URL '}'
        {
            $value = $URL.text;
        }
    ;   

anchor returns [String value]
    :   URL
        {
            $value = "<a href=\"" + $URL.text + "\">" + $URL.text + "</a>";
        }
    ;   

everythingElse returns [String value]
    :   CHUNK
        {
            $value = $CHUNK.text;
        }
    ;

LABEL
    :   ('a'..'z'|'A'..'Z') ('a'..'z'|'A'..'Z'|'0'..'9')*
    ;
URL
    :   'http://' ('a'..'z'|'A'..'Z'|'0'..'9'|'.'|'/'|'-'|'_'|'%'|'&'|'?')+
    ;
CHUNK
    //: (~('${'|'$video{'|'$image{'|'$audio{'))+
    :   ('a'..'z'|'A'..'Z'|'0'..'9'|'-'|' '|','|'.'|'?'|'\''|':'|'\t'|'\n'|'\r'|'\"'|'>'|'<'|'/'|'_'|'='|';'|'('|')'|'&')+
    ;

I am new to antlr - been using it for about a day, so please don't expect me to know a lot about it specifically.

Answer 1

I'm choosing this way to respond because of space restrictions and lack of formatting possibilities. So, anyway:

Nope. CHUNK and LABEL are Lexer (Scanner) rules, whereas fillInTheBlank is a parser rule. The parser operates on top of the scanner, that is, the scanner does not know about the parser (rules). You have to introduce lexer states:

First, you have to introduce an additional lexer member keeping the state, telling you whether you want to read a label or not:

@lexer::members {
    private boolean readLabel = false;
}

Then, you have to introduce explicit token definitions for '${' and '}' (I'm calling them BEGIN_VAR and END_VAR) which modify this state variable. Furthermore, END_VAR tokens can only be created when readLabel is true:

BEGIN_VAR
    : '${' { readLabel = true; };

END_VAR : { readLabel }?=> '}' { readLabel = false; };

You also have to tell the lexer that LABEL tokens, too, are to be generated only in this state:

LABEL
    :   { readLabel }?=> ('a'..'z'|'A'..'Z') ('a'..'z'|'A'..'Z'|'0'..'9')*
    ;

Note that it's crucial here that this definition appears before CHUNK.

Finally, you have to modify your fillInTheBlanks rule using the above token definitions:

fillInTheBlank returns [String value]
    :   BEGIN_VAR LABEL END_VAR 
        { ...

Hope this helps, seems to work for me.