CODEWITHSUNDEEP
bisonlexflex-lexer
Akusete
Akusete

Reputation: 10794

Separating reserved identifiers in flex

I have a language I am making a parser for which contains function calls. A few function names are reserved and I would like to handle them differently in my grammer. In EBNF it would look like

FunctionCall ::= FunctionName '(' ')'
SpecialFunctionCall :: SpecialName '(' ')'

FunctionName ::= VariableName - SpecialFunctionName

SpecialFunctionName ::= "special_function_a" | "special_function_b"

My problem is in translating the exception operator from EBNF to flex.

FunctionName    {Letter}{LetterOrDigit}

Is a super set of SpecialFunctionName, which is a hard-coded string

SpecialFunctionName   "special_function_a" | "special_function_b"

Hence I get a warning from bison saying that SpecialFunction will never be matched. Should I merge the tokens and compare the strings in the parser, or is there a recommended way to resolve this ambiguity in in flex?

Upvotes: 0

Views: 819

Answers (2)

Chris Dodd
Chris Dodd

Reputation: 126203

As long as you put the SpecialFunction rule FIRST in the lexer file:

{SpecialFunctionName}    { return SpecialName; }
{FunctionName}           { return FunctionName; }

any identifer that matches both patterns will trigger the first rule and thus return SpecialName instead of FunctionName.

Upvotes: 0

Jonathan Leffler
Jonathan Leffler

Reputation: 753725

The normal way of dealing with this to have the lexical analyzer recognize the special names and return the correct token type (SpecialName) for the special names and a regular identifier token (apparently FunctionName) for the other tokens.

However, it normally requires an undue degree of prescience on the part of the lexical analyzer to say that a particular (non-reserved, non-special) word is a function name rather than a simple identifier (which could also be a simple variable - unless you've gone down the Perl route of using sigils to identify variables from functions).

Upvotes: 3

Related Questions