Strange semantic behaviour of boost spirit x3 after splitting

Question

I came across a strange behaviour of boost spirit x3, after I splittet my grammar up into the recommended parser.hpp, parser_def.hpp, parser.cpp files. My example gramar parses some kind of easy enums:

enum = "enum" > identifier > "{" > identifier % "," > "}

this is my enum grammar. When I don't split the enum and identifier parser into the recommended files, everything works fine, especially the string "enum {foo, bar}" throws an expectation failure, as expected. This example can be found here: unsplitted working example

But when I split the exactly same grammar up into the different files, the parser throws

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid

trying to parse the same string "enum {foo, bar}"

this example can be found here: splitted strange example

1. ast.hpp

   #pragma once
   
   #include <vector>
   #include <string>
   #include <boost/fusion/include/adapt_struct.hpp>
   
   
   
   namespace ast{
   
   namespace x3 = boost::spirit::x3;
   
   struct Enum {
       std::string _name;
       std::vector<std::string> _elements;
   };
   
   
   }
   
   BOOST_FUSION_ADAPT_STRUCT(ast::Enum, _name, _elements)
   

2. config.hpp

   #pragma once 
   
   #include <boost/spirit/home/x3.hpp>
   
   namespace parser{
   
       namespace x3 = boost::spirit::x3;
   
       typedef std::string::const_iterator iterator_type;
       typedef x3::phrase_parse_context<x3::ascii::space_type>::type context_type;
   
   }
   

3. enum.cpp

   #include "enum_def.hpp"
   #include "config.hpp"
   
   namespace parser { namespace impl {
        BOOST_SPIRIT_INSTANTIATE(enum_type, iterator_type, context_type)
   }}
   
   namespace parser {
   
   const impl::enum_type& enum_parser()
   {
       return impl::enum_parser;
   }
   
   }
   

4. enum_def.hpp

   #pragma once
   
   #include "identifier.hpp"
   #include "enum.hpp"
   #include "ast.hpp"
   
   namespace parser{ namespace impl{
   
       namespace x3=boost::spirit::x3;
   
       const enum_type enum_parser = "enum";
   
       namespace{
           const auto& identifier = parser::identifier();
       }
       auto const enum_parser_def =
           "enum"
           > identifier
           > "{"
           > identifier % ","
           >"}";
   
       BOOST_SPIRIT_DEFINE(enum_parser)
   }}
   

5. enum.hpp

   #pragma once
   
   #include <boost/spirit/home/x3.hpp>
   #include "ast.hpp"
   
   namespace parser{ namespace impl{
       namespace x3=boost::spirit::x3;
   
       typedef x3::rule<class enum_class, ast::Enum> enum_type;
   
       BOOST_SPIRIT_DECLARE(enum_type)
   
   }}
   
   namespace parser{
       const impl::enum_type& enum_parser();
   }
   

6. identifier.cpp

   #include "identifier_def.hpp"
   #include "config.hpp"
   
   namespace parser { namespace impl {
        BOOST_SPIRIT_INSTANTIATE(identifier_type, iterator_type, context_type)
   }}
   
   namespace parser {
   
   const impl::identifier_type& identifier()
   {
       return impl::identifier;
   }
   
   }
   

7. identifier_def.hpp

   #pragma once
   #include <boost/spirit/home/x3.hpp>
   #include "identifier.hpp"
   
   namespace parser{ namespace impl{
   
       namespace x3=boost::spirit::x3;
   
       const identifier_type identifier = "identifier";    
   
       auto const identifier_def = x3::lexeme[
           ((x3::alpha | '_') >> *(x3::alnum | '_'))
       ];
   
       BOOST_SPIRIT_DEFINE(identifier)
   }}
   

8. identifier.hpp

   #pragma once
   #include <boost/spirit/home/x3.hpp>
   
   namespace parser{ namespace impl{
       namespace x3=boost::spirit::x3;
   
       typedef x3::rule<class identifier_class, std::string> identifier_type;
   
       BOOST_SPIRIT_DECLARE(identifier_type)
   }}
   
   
   namespace parser{
       const impl::identifier_type& identifier();
   }
   

9. main.cpp

   #include <boost/spirit/home/x3.hpp>
   #include "ast.hpp"
   #include "enum.hpp"
   
   namespace x3 = boost::spirit::x3;
   
   template<typename Parser, typename Attribute>
   bool test(const std::string& str, Parser&& p, Attribute&& attr)
   {
       using iterator_type = std::string::const_iterator;
       iterator_type in = str.begin();
       iterator_type end = str.end();
   
       bool ret = x3::phrase_parse(in, end, p, x3::ascii::space, attr);
       ret &= (in == end);
       return ret;
   
   }
   
   int main(){
       ast::Enum attr;
       test("enum foo{foo,bar}", parser::enum_parser(), attr);
       test("enum {foo,bar}", parser::enum_parser(), attr);    
   }
   

Is this a bug, am I missing something, or is this an expected behaviour?

EDIT: here is my repo with an example which throws an std::logic_error instead of an expectation_failure

Answer 1

Here's a solution that worked for me, when the above workround does not.

Say you have files a.cpp, a.h, a_def.hpp, b.cpp, b.h, b_def.hpp, ... as recommended by the Boost.Spirit X3 docs.

The basic idea is to combine the *.cpp and *_def.hpp files into 1 file for each group. The *.h files can and should remain.

1. ls *_def.hpp > parser_def.hpp (assuming parser_def.hpp doesn't already exist) and edit parser_def.hpp to #include the files in the correct order. Remove redundant lines (add a header guard, etc.) The goal is for parser_def.hpp to include the other files in the correct order.
2. cat *.cpp > parser.cpp and edit parser.cpp to be syntactically correct, and replace all #include <*_def.hpp> lines with a single #include <parser_def.hpp> at the top. In your build file (e.g. make or cmake), replace the compilation of the *.cpp files with the single parser.cpp.
3. You can get rid of the old *.cpp files.

You lose the convenience of separately compiled files, but it will avoid the Static Initialization Order Fiasco.

Answer 2

I've found the cause of the bug.

The bug is with the fact that the expect directive takes it subject parser by value, which is before the parser::impl::identifier initializer runs.

To visualize, imagine the static initializer for parser::impl::enum_parser running before parser::impl::identifier. This is valid for a compiler to do.

The copy, therefore, has an uninitialized name field, which fails as soon as the expectation point tries to construct the x3::expectation_failure with the which_ member, because constructing a std::string from a nullptr is illegal.

All in all, I fear the root cause here is Static Initialization Order Fiasco. I'll see whether I can fix it and submit a PR.

WORKAROUND:

An immediate workaround is to list the order of the source files in reverse, so that use comes after definition:

set(SOURCE_FILES 
    identifier.cpp
    enum.cpp 
    main.cpp 
)

Note that if this fixes it on your compiler (it does on mine) that is implementation defined. The standard does NOT specify the order of static initialization across compilation units.