Error using python and the Ply parser generator

Question

I am using Ply for teaching and I really like it. I though to use decorators for not repeating some code that I want in some functions. So, I try to use the following code:

import ply.yacc as yacc
from functools import wraps
from CLexer import Lexico


def producciones(function):
    """
    Decorator for each of the functions which represents
    grammatical rules.
    """
    variable = function.__doc__.split(':')[0].strip()
    @wraps(function)
    def wrapper(*args,**kargs):
        result = []
        for e in args[1][1:]:
            tmp = Node()
            if isinstance(e,Node):
                tmp = e 
            else:
                tmp.type = str(e)
            result.append(tmp)
        tmp = Node(result)
        tmp.type = variable
        args[1][0] = tmp
        function(*args, **kargs)
        return wrapper


class Sintaxis:

    tokens = Lexico.tokens
    start = 'programa'
    @producciones
    def p_program(self, p):
        """
        program : ABREPAREN program CIERRAPAREN program
        | 
        """



    def p_error(self, p):
        print("Syntax error at '%s'" % p.value)

    def run(self, s):
        lexico = Lexico()
        lexico.build()
        global tokens
        self.parser = yacc.yacc(debug = True, module= self)
        result =self.parser.parse(s,lexico)
        return  result




if __name__ == '__main__':
    with open("prueba.txt") as f:
        texto=f.read()
    parser = Sintaxis()
    result = parser.run(texto)

My problem is when trying to use decorators, which gives the following error:

ERROR: new.py:15: Rule 'p_program' requires an argument

I haven't found this error in the documentation, and the method p_program seems to accept two arguments... Any clues? Thank you for your help.

Answer 1

Lets work through your problem as a tutorial in tracking down the source of bugs. First lets enumerate the number of differences between your example code and the documentation for PLY:

1. You have used decorators on the syntax rule functions
2. You have used two arguments to syntax rule functions
3. You have started the syntax rule with a blank line
4. You have used the """ string notation of python - not used in the manual
5. You have used empty production in a manner deprecated in the manual
6. You have specified a start rule of programa which does not exist

This could be solved by writing example programs which test each one individually and showing that they work or not, then by elimination we can deduce why you have a fault.

Fortunately the PLY distribution contains several working examples which can be consulted as an adjunct to the manual, and there are some linked from the PLY homepage. One of those examples shows rules specified using the """ syntax, and also uses two arguments for a syntax rule function; that eliminates reasons 2 & 4:

def p_declaration(self, p):
        """ declaration : decl_body SEMI
        """
        p[0] = p[1]

If we examine more of the examples supplied with the PLY distribution we can find example\classcalc\calc.py, which has the rule that starts with a blank line and uses the """ syntax and also has two arguments, eliminated reasons 2, 3 & 4:

def p_expression_binop(self, p):
        """
        expression : expression PLUS expression
                  | expression MINUS expression
                  | expression TIMES expression
                  | expression DIVIDE expression
                  | expression EXP expression
        """

We need to eliminate the empty production notation issue. Using a grep on all the example programs in the distribution reveals one using a blind production. This is the BASIC interpreter. Here (in the file examples\BASIC\basparse.py) we have the rule:

def p_optstep(p):
    '''optstep : STEP expr
               | empty'''
    if len(p) == 3:
       p[0] = p[2]
    else:
       p[0] = None

This rule shows the recommended way of specifying a blind production, however the manual does say:

Note: You can write empty rules anywhere by simply specifying an empty right hand side. However, I personally find that writing an "empty" rule and using "empty" to denote an empty production is easier to read and more clearly states your intentions.

If we rewrite this rule to match your style, we can test the hypothesis:

def p_optstep(p):
    '''optstep : STEP expr
               | '''
    if len(p) == 3:
       p[0] = p[2]
    else:
       p[0] = None

The experiment shows that the above code still works, and thus eliminating reason 5. We are now left with reasons 1 & 6. Reason 6 is easy for you to eliminate in your code, and we are left with reason 1. Further, a grep for decorators in all available working PLY examples does not show any. This means that no one, even the experts, have used decorators in the manner shown in your code. I suspect there is a reason for that. They don't work. The only use of decorators in PLY is with tokens that have complex regular expressions.

Conclusion

1. Stop using decorators
2. Fix the start symbol programa
3. Supply the necessary information when asking for help

---

[*] Where is lexico and CLexer, where is prueba.txt? Do I have to define my own tokens and write my own lexer and guess what text to parse? No, you should help me with that.