Bison/Flex Parsing File

Question

I have recently tried using GNU Bison and Flex to write a interpreter. The text I want the interpreter to recognize is print "Hello" and I have tried the following:

flex file:

%{
#include <iostream>
using namespace std;
#define YY_DECL extern "C" int yylex()
#include "gbison.tab.h"
%}
%%
[ \t\n]            ;
'\"'               return QUOTE;
[a-zA-Z0-9]+       { yylval.sval = strdup(yytext); return STRING; }
%%

bison file:

%{
#include <cstdio> 
#include <cstring>
#include <iostream>
using namespace std;

extern "C" int yylex();
extern "C" int yyparse();
extern "C" FILE* yyin;

void yyerror (const char* s);
%}

%union {
  char* sval;
}

%token <sval> STRING
%token QUOTE
%%

str:
    STRING QUOTE STRING QUOTE
    {
       if (strcmp($1, "print") == 0)
       {
           cout << $3 << flush;
       }
       if (strcmp($1, "println") == 0)
       {
           cout << $3 << endl; 
       }
    }
    ;
%%

main(int argc, char* argv[])
{
   FILE* input = fopen(argv[1], "r");
   if (!input)
   {
      cout << "Bad input. Nonexistant file" << endl; 
      return -1;
   } 

   yyin = input;

   do 
   {
       yyparse();
   } while (!feof(yyin));

}
void yyerror(const char* s)
{
   cout << "Error. " << s << endl; 
   exit(-1);   
}

But when I pass print "hello" to the compiled program I get: "Error. syntax error

I think that the issue is the STRING QUOTE STRING QUOTE but I am not sure. What is exactly is going wrong? How would I get the interpreter to print hello?

Answer 1

The answers are below, but I hope the following is more generally useful, as fishing instruction.

There are a variety of debugging tools which would help you. In particular, flex provides the -d flag:

-d, --debug

makes the generated scanner run in "debug" mode. Whenever a pattern is recognized and the global variable yy_flex_debug is non-zero (which is the default), the scanner will write to stderr a line… (flex manual)

bison also provides a debug facility. (bison manual)

There are several means to enable compilation of trace facilities:

• the macro YYDEBUG…
• the option -t (POSIX Yacc compliant)…
• the option --debug (Bison extension)…
• the directive %debug…

We suggest that you always enable the debug option so that debugging is always possible.
…
Once you have compiled the program with trace facilities, the way to request a trace is to store a nonzero value in the variable yydebug. You can do this by making the C code do it (in main, perhaps), or you can alter the value with a C debugger.

Also, remember that flex inserts an automatic rule which causes any otherwise unrecognized character to be echoed to the output. ("By default, any text not matched by a flex scanner is copied to the output" -- Some simple examples) That's why you have the extra " in the error message being printed by your program:

"Error. syntax error
^

That's a bit subtle, though. Tracing flex would have shown you that more directly.

So, finally, the problem(s):

1. The flex pattern '\"' does not match a ". It matches '"', because single quotes are not special to flex. That's definitely why your parse fails.

2. Fixing that will let your program parse a single command, but it will generate a syntax error if you try to give it two print commands in the same input. That's because bison always parses until it receives an END token from the lexer, and the lexer (by default) only provides an END token when it reaches the end of the input. You can change

   • the lexer behaviour (by sending END in other circumstances, for example a new-line) (not recommended)

   • the parser behaviour (by using ACCEPT) (possible, but rarely necessary)

   • the grammar, so that it recognizes any number of statements. (recommended)