class istream in Flex Lexer Tool

Question

I am facing some issue in flex with 2.5.4 version and not able to figure out issue. In recent, Rhel machine has upgraded to 6.5 from 5.

1. Can I replace class istream with #include <isotream> using namespace std;
2. if yes, what should I write into lexer.l file.
3. In flex-2.5.3 it generate #include<iostream> using namespace std only
4. I am not able to use flex-2.5.3 as it does not support RHEL5 machine.

The lexer.l (modify to create simple test case)

{
/* need this for the call to atof() below */
#include <math.h>
#include <string>
#include <iostream>

#undef yyFlexLexer
#define yyFlexLexer hSpiceConverterFlexer
#include "kernel.h"
using namespace std;
static bool a = true;
%}

%option c++ debug

%%

%{
string copyText;
%}

%%

The generated scanner file(The file is big so adding problem part)

/* A lexical scanner generated by flex */

/* Scanner skeleton version:
 * $Header: /home/daffy/u0/vern/flex/RCS/flex.skl,v 2.91 96/09/10 16:58:48 vern Exp $
 */

#define FLEX_SCANNER
#define YY_FLEX_MAJOR_VERSION 2
#define YY_FLEX_MINOR_VERSION 5



/* cfront 1.2 defines "c_plusplus" instead of "__cplusplus" */
#ifdef c_plusplus
#ifndef __cplusplus
#define __cplusplus
#endif
#endif


#ifdef __cplusplus

#include <stdlib.h>
class istream;
#include <unistd.h>

if class istream replace with <iostream> using namespace std; I am able to compile with gcc

The error which I get with class istream

ex.yy.cc: In member function ¡virtual int hSpiceConverterFlexer::yylex()¢:
lex.yy.cc:987:37: error: cannot convert ¡std::istream* {aka std::basic_istream<char>*}¢ to ¡istream*¢ in assignment
lex.yy.cc: At global scope:
lex.yy.cc:1095:25: error: expected constructor, destructor, or type conversion before ¡(¢ token
lex.yy.cc:1130:35: error: variable or field ¡switch_streams¢ declared void
lex.yy.cc:1130:35: error: reference to ¡istream¢ is ambiguous
lexer.cpp:24:7: note: candidates are: class istream
 class istream;
       ^
In file included from /depotbld/RHEL6.0/gcc-4.8.2/include/c++/4.8.2/ios:38:0,
                 from /depotbld/RHEL6.0/gcc-4.8.2/include/c++/4.8.2/ostream:38,
                 from /depotbld/RHEL6.0/gcc-4.8.2/include/c++/4.8.2/iostream:39,
                 from ../include/FlexLexer.h:47,
                 from lexer.cpp:239:
/depotbld/RHEL6.0/gcc-4.8.2/include/c++/4.8.2/iosfwd:133:33: note:                 typedef class std::basic_istream<char> std::istream
   typedef basic_istream<char>   istream;
                                 ^
lex.yy.cc:1130:44: error: ¡new_in¢ was not declared in this scope
lex.yy.cc:1130:59: error: expected primary-expression before ¡*¢ token
lex.yy.cc:1130:61: error: ¡new_out¢ was not declared in this scope
lexer.cpp:209:14: warning: ¡void* yy_flex_alloc(yy_size_t)¢ declared ¡static¢ but never defined [-Wunused-function]
 static void *yy_flex_alloc YY_PROTO(( yy_size_t ));
              ^
lexer.cpp:210:14: warning: ¡void* yy_flex_realloc(void*, yy_size_t)¢ declared ¡static¢ but never defined [-Wunused-function]
 static void *yy_flex_realloc YY_PROTO(( void *, yy_size_t ));
              ^
lexer.cpp:211:13: warning: ¡void yy_flex_free(void*)¢ declared ¡static¢ but never defined [-Wunused-function]
 static void yy_flex_free YY_PROTO(( void * ));

Answer 1

It's unlikely that the problem was caused by the change to flex 2.5.4. The differences between 2.5.3 and 2.5.4 were minimal. Much more likely is that neither flex 2.5.3 nor 2.5.4 is compatible with the version of the standard C++ library included in RHEL 6.5

As I understand it, the version of flex for RHEL 6.5 is 2.5.35, which is a lot closer to modern even though it is far from the most recent version. Upgrading to that version should fix the problem, since it uses a more normal set of C++ includes.

---

After some experimentation, I successfully compiled a simple flex file with the C++ API using gcc-4.8 and flex 2.5.4. To do so, I created a slightly-modified flex.skl file using the following patch file:

--- flex.skl    1996-09-10 18:58:54.000000000 -0500
+++ flex.skl.new        2016-01-05 23:16:36.435900415 -0500
@@ -8,9 +8,7 @@
 #define YY_FLEX_MAJOR_VERSION 2
 #define YY_FLEX_MINOR_VERSION 5

-%-
 #include <stdio.h>
-%*


 /* cfront 1.2 defines "c_plusplus" instead of "__cplusplus" */
@@ -25,7 +23,8 @@

 #include <stdlib.h>
 %+
-class istream;
+#include <iostream>
+using namespace std;
 %*
 #include <unistd.h>

Also, the FlexLexer.h file included in the 2.5.4 distribution references iostream.h. You'll probably need to change that to iostream.

With that done, I could generate the C++ file with:

flex-2.5.4 -Sflex.skl.new ...

As can be seen from the patch above, the fix consists of:

• always including stdio.h. If you don't do that, EOF is not defined.
• removing the forward declaration of istream and replacing it with #include <iostream>. Since that header will be included anyway, it seems like it should be OK to include it earlier.
• adding a using namespace std;, which matches the usage in flex 2.5.4. Apparently, the C++ standard library it was designed to use did not put standard prototypes into the std namespace.

I still think that using ancient flex versions is not a good idea, and I encourage you to upgrade to the latest stable version (currently 2.5.39), or at least to 2.5.35. If you encounter some problem with that, try to create a Minimal Compilable example and submit a separate question.