CODEWITHSUNDEEP
flex-lexer
adhanlon
adhanlon

Reputation: 6539

difficulty getting c-style comments in flex/lex

I want to make a rule in flex to consume a c-style comment like /* */

i have the following

c_comment "/*"[\n.]*"*/"

But it doesn't ever get matched. Any idea why? if you need more of my code please let me know and I'll submit the whole thing. Thanks to anyone who replies.

Upvotes: 30

Views: 33065

Answers (9)

lindyang
lindyang

Reputation: 1

ignore the space and newline

"/*"
  (
    "/"*
      (
        "*"*
        [^*/]
        "/"*
      )*
    "*"*
  )*
"*/"

Kenneth C. Louden - Compiler Construction_ Principles and Practice (1997) section 2.2.3

Upvotes: 0

TayebDIAB
TayebDIAB

Reputation: 11

Another example:

"/*"([^*]*|(\*+[^/]))*"*/"

Upvotes: 1

zneak
zneak

Reputation: 138051

I suggest you use start conditions instead.

%x C_COMMENT

"/*"            { BEGIN(C_COMMENT); }
<C_COMMENT>"*/" { BEGIN(INITIAL); }
<C_COMMENT>\n   { }
<C_COMMENT>.    { }

Do note that there must not be any whitespace between the <condition> and the rule.

%x C_COMMENT defines the C_COMMENT state, and the rule /* has it start. Once it's started, */ will have it go back to the initial state (INITIAL is predefined), and every other characters will just be consumed without any particular action. When two rules match, Flex disambiguates by taking the one that has the longest match, so the dot rule does not prevent */ from matching. The \n rule is necessary because a dot matches everything except a newline.

The %x definition makes C_COMMENT an exclusive state, which means the lexer will only match rules that are "tagged" <C_COMMENT> once it enters the state.

Here is a tiny example lexer that implements this answer by printing everything except what's inside /* comments */.

Upvotes: 48

David Given
David Given

Reputation: 13701

There's a worked example in the Flex manual, which gets the gnarly edge cases right:

<INITIAL>"/*"         BEGIN(IN_COMMENT);
<IN_COMMENT>"*/"      BEGIN(INITIAL);
<IN_COMMENT>[^*\n]+   // eat comment in chunks
<IN_COMMENT>"*"       // eat the lone star
<IN_COMMENT>\n        yylineno++;

Upvotes: 1

Andrey
Andrey

Reputation: 9

The worked example is:

\/\*([^*]|[\r\n]|(\*+([^*/]|[\r\n])))*\*+\/

which found in ostermiller.org

Upvotes: 0

mwag
mwag

Reputation: 4035

I've tried several of the suggested solutions and here are the results.

  • I could not get the C_COMMENT solution, which has the most up-votes and looks great, to work at all in practice (one of the comments to it explains at least one reason why). It should be downvoted and certainly should not be the highest-voted solution
  • The solution from Mugen seemed to work in all of the code I ran it on
  • Could not get the solution from Andrey to even compile at all in lex. I looked at the referenced website and using patterns from there did not help

  • the answer from paxdiablo worked and had the advantage of being easy to read. I further modified as follows:

    "/*" { int c1 = 0, c2 = input();
       for(;;) {
         if(c2 == EOF) break;
         if(c1 == '*' && c2 == '/')
           break;
         c1 = c2;
         c2 = input();
       }
     }

Upvotes: 1

Mugen
Mugen

Reputation: 9095

I believe this solution is simpler:

"/*"((\*+[^/*])|([^*]))*\**"*/"

Upvotes: 2

user1747935
user1747935

Reputation: 362

Here's an example just in case anyone is confused about how to work zneak's answer:

(Basically, you put "%x C_COMMENT" in the first section and the rest in the second section, as explained by his helpful link)

foo.l

%{
// c code..
%}
%x C_COMMENT

%%
"/*"            { BEGIN(C_COMMENT); }
<C_COMMENT>"*/" { BEGIN(INITIAL); }
<C_COMMENT>.    { }

%%
// c code..

Hope that helps someone! Tiff

Upvotes: 9

paxdiablo
paxdiablo

Reputation: 881403

Not sure why it's not being picked up but I do know that a pattern of that sort can produce large lexical elements. It's more efficient to detect just the start comment marker and toss everything in the bitbucket until you find the end marker.

This site has code which will do that:

"/*" {
    for (;;) {
        while ((c = input()) != '*' && c != EOF)
            ; /* eat up text of comment */
        if (c == '*') {
            while ((c = input()) == '*')
                ;
            if (c == '/')
                break; /* found the end */
        }
        if (c == EOF) {
            error ("EOF in comment");
            break;
        }
    }
}

Upvotes: 7

Related Questions