CODEWITHSUNDEEP
pythonparsingply
Paula Moraes
Paula Moraes

Reputation: 13

PLY - Parsing error when adding a second similar line

I was testing ply with a simple .txt file containing: value = 0.4. And the parser worked as expected, but when I added a second line to this file I've got an error:

Error: syntax error when parsing 'LexToken(VALUE_KEY,'value',1,15)'

My parser:

from ply import lex
from ply import yacc
from sys import argv

tokens = ('PROBABILITY',
          'EQUALS',
          'VALUE_KEY',
          'NAME')

t_ignore = ' \t'
t_EQUALS = r'='

reserved = {'value' : 'VALUE_KEY'}

def t_KEYWORD(t):
    r':?[a-zA-z_][a-zA-Z_0-9\-]*'
    t.type = reserved.get(t.value, 'NAME')
    return t

def t_NAME(t):
    r'[a-zA-z_][a-zA-Z_0-9\-]*'
    return t

def t_PROBABILITY(t):
    r'[0-1]\.\d+'
    t.value = float(t.value)
    return t

def t_newline(t):
    r'\n+'
    t.lineno += len(t.value)

def t_error(t):
    print("Error: illegal character '{}'".format(t))
    t.lexer.skip(1)


# build the lexer
lex.lex()

def p_prob_def(p):
    '''prob_def : VALUE_KEY EQUALS PROBABILITY'''
    p[0] = p[3]

def p_error(p):
    print("Error: syntax error when parsing '{}'".format(p))

# build parser
yacc.yacc()

class ToyParser(object):

    @classmethod
    def parse(cls, filename):
        with open(filename, 'r') as file:
            data = ''
            for line in file:
                data += '\n' + line
        return yacc.parse(data)

if __name__=='__main__':
    test = ToyParser.parse(argv[1])
    print(test)

Input file that produced the error:

value = 0.4 
value = 0.7

Upvotes: 1

Views: 241

Answers (1)

rici
rici

Reputation: 241731

Your grammar only recognizes a single prob_def. Ply parsers, like those produced by many parser generators, insists that the starting symbol in the grammar matches the entire input (otherwise, trailing garbage wouldn't be correctly recognised as an error).

If you want the grammar to parse multiple objects, you need to write an explicit rule:

def p_empty(p):
    '''program :'''
    p[0] = []

def p_program(p): 
    '''program : program prob_def '''
    p[0] = p[1]
    p[0].append(p[2])

Those need to go before the definition of prob_def, so that program becomes the start symbol.

Upvotes: 1

Related Questions