CODEWITHSUNDEEP
pythontatsu
Patrick
Patrick

Reputation: 187

Matching the hash character in Tatsu

I am getting an exception attempting to parse the # character using Tatsu:

import tatsu

grammar = r'''
@@comments :: //
@@eol_comments :: //

start = '#' ;
'''

print(tatsu.__version__)

parser = tatsu.compile(grammar)
ast = parser.parse('#', trace=True)

5.8.3
↙start ~1:1
#
≢'#' 
≢start ~1:1
#

...

tatsu.exceptions.FailedToken: (1:1) expecting '#' :
#
^
start

If I change the # to a in both the grammar and the text then it is successful. I think the issue might be that # indicates a grammar comment in Tatsu, but I'm not sure how to escape it.

Upvotes: 2

Views: 61

Answers (2)

Apalala
Apalala

Reputation: 9244

I researched this problem today for a long while, and found much room for improvement in TatSu.

Yet the conclusion is that // is a valid regex that matches zero input.

The solution in your case is to set the comments to strings you don't expect to find in the input:

    grammar = r'''
        @@comments :: /@@@@@@@/
        @@eol_comments :: /@@@@@@@/

        start = '#' ;
    '''

https://github.com/neogeny/TatSu/issues/303

Upvotes: 0

Apalala
Apalala

Reputation: 9244

The problem here is that config.eol_comments_re is not being overridden with the @@eol_comments definition.

Could you post an issue for this problem at https://github.com/neogeny/TatSu/issues?

The other problem is that comments should not be checked while parsing a '' token.

Upvotes: 0

Related Questions