CODEWITHSUNDEEP
antlrantlr3
rickythefox
rickythefox

Reputation: 6851

ANTLR - allowing for incomplete grammar

I'm using ANTLR to parse strings of mathematical expressions and tag them using MathML.

Right now I have the grammar below. Now I have three questions:

  1. The grammar allows for complete expressions like 2*(3+4). I want it to also allow incomplete expressions, e.g. 2*(3+. Being a complete newbie at ANTLR I have no idea how to accomplish this. Please point me to the right document or give an example.
  2. The location of the square root rule sqrt among the atomics seems to work but I'm pretty sure it should be somewhere in the exponent rule? Or should it?
  3. If I want to extend this grammar to also actually perform the calculation, can I somehow reuse it or do I have to copy and paste?

Any other comments or suggestions on my grammar is also appreciated, as my total experience with ANTLR is now about four hours.

grammar Expr;

parse returns [String value]
    :   stat+ {$value = $stat.value;}
    ;

stat returns [String value]
    :   exponent NEWLINE {$value = "<math>" + $exponent.value + "</math>";}
    |   NEWLINE
    ;

exponent returns [String value]
    :   e=expr {$value = $e.value;}
        (   '^' e=expr {$value = "<msup><mrow>" + $value + "</mrow><mrow>" + $e.value + "</mrow></msup>";}
        )*
    ;

expr returns [String value]
    :   e=multExpr {$value = $e.value;}
        (   '+' e=multExpr {$value += "<mo>+</mo>" + $e.value;}
        |   '-' e=multExpr {$value += "<mo>-</mo>" + $e.value;}
        )*
    ;

multExpr returns [String value]
    :   e=atom {$value = $e.value;} 
        (   '*' e=atom {$value += "<mo>*</mo>" + $e.value;}
        |   '/' e=atom {$value += "<mo>/</mo>" + $e.value;}
        )*
    ; 

atom returns [String value]
    :   INT {$value = "<mn>" + $INT.text + "</mn>";}
    |   '-' e=atom {$value = "<mo>-</mo>" + $e.value;}
    |   'sqrt[' exponent ']' {$value = "<msqrt><mrow>" + $exponent.value + "</mrow></msqrt>";}
    |   '(' exponent ')' {$value = "<mo>(</mo>" + $exponent.value + "<mo>)</mo>";}
    ;

INT :   '0'..'9'+ ;
NEWLINE:'\r'? '\n' ;
WS  :   (' '|'\t')+ {skip();} ;

Upvotes: 2

Views: 627

Answers (1)

Bart Kiers
Bart Kiers

Reputation: 170148

First a few remarks about your grammar:

  • you should give the rules unique labels for left- and right hand sides (e1=atom ('*' e2=atom ...);
  • you'll probably want to create separate sqrt and [ tokens instead of 1 single sqrt[, otherwise input like "sqrt [ 9 ]" (a space between sqrt and [) would not be handles properly;
  • unary minus usually has a lower precedence than exponentiation.

rickythefox wrote:

The location of the square root rule sqrt among the atomics seems to work but I'm pretty sure it should be somewhere in the exponent rule? Or should it?

No, it's fine there: it should have the highest precedence. Talking of precedence, the usual precedence table (from lowest to highest) in your case would be:

  • addition & subtraction;
  • multiplication & division;
  • unary minus;
  • exponentiation;
  • parenthesized expressions (including function calls, like sqrt[...]).

rickythefox wrote:

The grammar allows for complete expressions like 2*(3+4). I want it to also allow incomplete expressions, e.g. 2*(3+. Being a complete newbie at ANTLR I have no idea how to accomplish this. Please point me to the right document or give an example.

That's tricky.

I really only see one way: inside your stat rule, you first force the parser to look ahead in the token stream to check if there really is an expr ahead. This can be done using a syntactic predicate. Once the parser is sure there is an expr, only then parse said expression. If there isn't an expr, try to match a NEWLINE, and if there's also no NEWLINE, simply consume a single token other than NEWLINE (which must be a part of an incomplete expression!). (I will post a small demo below)

rickythefox wrote:

If I want to extend this grammar to also actually perform the calculation, can I somehow reuse it or do I have to copy and paste?

ANTLR parser rules can return more than one object. That's not really true of course since Java methods (which parser rule essentially are) can only return a single object. Parser rule return an object that holds references to more than one object. So you could do:

stat returns [String str, double num]
  :  ...
  ;

A demo

Taking all my hints into account, a small working demo could look like this:

grammar Expr;

parse returns [String str, double num]
@init{$str = "";}
  :  (stat 
     {
       $str += $stat.str;
       $num = $stat.num;
       if(!Double.isNaN($num)) {
         System.out.println($stat.text.trim() + " = " + $num);
       }
     })+
  ;

stat returns [String str, double num]
  : (expr)=> expr NEWLINE      {$str = "<math>" + $expr.str + "</math>"; $num = $expr.num;}
  |          NEWLINE           {$str = ""; $num = Double.NaN;}
  |          ~NEWLINE          {$str = ""; $num = Double.NaN; System.err.println("Ignoring: " + $text);}
  ;

expr returns [String str, double num]
  :  e1=multExpr       {$str = $e1.str; $num = $e1.num;}
     ( '+' e2=multExpr {$str += "<mo>+</mo>" + $e2.str; $num += $e2.num;}
     | '-' e2=multExpr {$str += "<mo>-</mo>" + $e2.str; $num -= $e2.num;}
     )*
  ;

multExpr returns [String str, double num]
  :  e1=unaryExpr       {$str = $e1.str; $num = $e1.num;} 
     ( '*' e2=unaryExpr {$str += "<mo>*</mo>" + $e2.str; $num *= $e2.num;}
     | '/' e2=unaryExpr {$str += "<mo>/</mo>" + $e2.str; $num /= $e2.num;}
     )*
  ; 

unaryExpr returns [String str, double num]
  :  '-' e=expExpr {$str = "<mo>-</mo>" + $e.str; $num = -1 * $e.num;}
  |  e=expExpr     {$str = $e.str; $num = $e.num;}
  ;

expExpr returns [String str, double num]
  :  e1=atom       {$str = $e1.str; $num = $e1.num;}
     ( '^' e2=atom {$str = "<msup><mrow>" + $str + "</mrow><mrow>" + $e2.str + "</mrow></msup>"; $num = Math.pow($num, $e2.num);}
     )*
  ;

atom returns [String str, double num]
  :  INT                 {$str = "<mn>" + $INT.text + "</mn>"; $num = Double.valueOf($INT.text);}
  |  'sqrt' '[' expr ']' {$str = "<msqrt><mrow>" + $expr.str + "</mrow></msqrt>"; $num = Math.sqrt($expr.num);}
  |  '(' expr ')'        {$str = "<mo>(</mo>" + $expr.str + "<mo>)</mo>"; $num = $expr.num;}
  ;

INT     : '0'..'9'+;
NEWLINE : '\r'? '\n';
WS      : (' '|'\t')+ {skip();};

(note that the (...)=> is this so-called syntactic predicate)

You can test the parser generated from the grammar above with the following class:

import org.antlr.runtime.*;

public class Main {
  public static void main(String[] args) throws Exception {
    String src =
        "sqrt [ 9 ] \n" +  
        "1+2*3      \n" + 
        "2*(3+      \n" +
        "2*(3+42)^2 \n";
    ExprLexer lexer = new ExprLexer(new ANTLRStringStream(src));
    ExprParser parser = new ExprParser(new CommonTokenStream(lexer));
    ExprParser.parse_return returnValue = parser.parse();
    String mathML = returnValue.str;
    double eval = returnValue.num;
    // ...
  }
}

And if you now run the class above, you will see that the input

sqrt [ 9 ]
1+2*3
2*(3+
2*(3+42)^2

will produce the following output:

sqrt[9] = 3.0
1+2*3 = 7.0
Ignoring: 2
Ignoring: *
Ignoring: (
Ignoring: 3
Ignoring: +
2*(3+42)^2 = 4050.0

Upvotes: 3

Related Questions