CODEWITHSUNDEEP
pythonpyparsing
whereswalden
whereswalden

Reputation: 4959

pyparsing Optional() & Optional() allows repetitions

I've this simple grammar:

word = Word(alphanums + '_')
with_stmt = Suppress('with') + OneOrMore(Group(word('key') + Suppress('=') + word('value')))('overrides')
using_stmt = Suppress('using') + Regex('id-[0-9a-f]{8}')('id')
modifiers = Optional(with_stmt('with_stmt')) & Optional(using_stmt('using_stmt'))
pattern = StringStart() + modifiers + StringEnd()

It seems that the Optional() & Optional() erroneously allows for multiple repetitions of either modifier, and only labels the last one:

>>> print dict(pattern.parseString('with foo=bar bing=baz using id-deadbeef using id-feedfeed'))
{
  'with_stmt': (
    [
      (['foo', 'bar'], {'value': [('bar', 1)], 'key': [('foo', 0)]}), 
      (['bing', 'baz'], {'value': [('baz', 1)], 'key': [('bing', 0)]})
    ],
    {'overrides': 
      [(([
        (['foo', 'bar'], {'value': [('bar', 1)], 'key': [('foo', 0)]}),
        (['bing', 'baz'], {'value': [('baz', 1)], 'key': [('bing', 0)]})
      ], {}), 0)]
    }
  ), 
  'overrides': 
    (
      [(['foo', 'bar'], {'value': [('bar', 1)], 'key': [('foo', 0)]}),
      (['bing', 'baz'], {'value': [('baz', 1)], 'key': [('bing', 0)]})], {}
    ),
  'id': (['id-deadbeef', 'id-feedfeed'], {}),
  'using_stmt': (['id-deadbeef', 'id-feedfeed'], {'id': [('id-deadbeef', 0), ('id-feedfeed', 1)]})
}

using_stmt matches both id-deadbeef and id-feedfeed instead of throwing an error at using id-feedfeed.

Strangely enough, if make the modifiers non-optional, then the repitition problem goes away and parsing fails as expected:

>>> dict(pattern.parseString('with foo=bar bing=baz using id-deadbeef using id-feedfeed'))
Traceback (most recent call last):
  File "parse.py", line 10, in <module>
    print dict(pattern.parseString('with foo=bar bing=baz using id-deadbeef using id-feedfeed'))
  File "/path/to/lib/python2.7/site-packages/pyparsing.py", line 1139, in parseString
    raise exc
pyparsing.ParseException: Expected end of text (at char 40), (line:1, col:41)

Switching to + instead of & also causes it to fail as expected. with_stmt exhibits the same problem, and making it non-optional also fixes it.

What is it about marking a pattern as optional that allows for repetition inside an Each()?

Upvotes: 3

Views: 213

Answers (1)

PaulMcG
PaulMcG

Reputation: 63709

This a bug in pyparsing's Each class - will be fixed in 2.0.6.

Upvotes: 1

Related Questions