Reputation: 1390
In a project I'm working on in ANTLR (v4), I have a rule like this:
Equal : '::='
Identifier
: ('A'..'Z' | '_' | '0'..'9') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*
;
Constant
: Identifier Equal Expression
;
And I give it code like:
X ::= 1
, and, instead of it being a "Constant", it comes out as three separate tokens: An Identifier, an Equal, and an Identifier. I can't seem to figure out what I am doing wrong. Maybe I should just go back to using Haskell's Parsec library. (The reason I stopped using that is a long story.)
Upvotes: 0
Views: 51
Reputation: 170308
They're 3 tokens because you have spaces in between. When you do X::=1
, it'd end up as a single Constant
token.
You may have a lexer rule defined that skips space-chars, but that will only cause the parser to never see these space-chars, not the lexer.
Your constant rule should be a parser rule, then you won't have any issues. It doesn't make much sense to create 1 single constant token. So should your Expression rule, presumably. Tokens (lexer rules) are the fundamental building blocks of your language. Like atoms in chemistry. Parser rules (lower cased rules) are used to glue together these tokens.
Upvotes: 1