How to make recursive Spirit X3 parser with a separate visitor class

Question

A parser application where I’m working on calls for recursive rules. Besides looking into the Recursive AST tutorial examples of Boost Spirit X3 which can be found here: https://www.boost.org/doc/libs/develop/libs/spirit/doc/x3/html/index.html, I was looking for a solution with a std::variant of some types as well as a std::vector of that same variant type.

In the StackOverflow post titled: Recursive rule in Spirit.X3, I found the code from the answer from sehe a decent starting point for my parser.

I have repeated the code here but I have limited the input strings to be tested. Because the full list from the original is not relevant for this question here.

//#define BOOST_SPIRIT_X3_DEBUG
#include <iostream>
#include <boost/fusion/adapted.hpp>
#include <boost/spirit/home/x3.hpp>
#include <string>
#include <vector>
#include <variant>

struct value: std::variant<int,float,std::vector<value>>
{
    using base_type = std::variant<int,float,std::vector<value>>;
    using base_type::variant;

    friend std::ostream& operator<<(std::ostream& os, base_type const& v) {
        struct {
            std::ostream& operator()(float const& f) const { return _os << "float:" << f; }
            std::ostream& operator()(int const& i)   const { return _os << "int:" << i; }
            std::ostream& operator()(std::vector<value> const& v) const {
                _os << "tuple: [";
                for (auto& el : v) _os << el << ",";
                return _os << ']';
            }
            std::ostream& _os;
        } vis { os };

        return std::visit(vis, v);
    }
};

namespace parser {
    namespace x3 = boost::spirit::x3;

    x3::rule<struct value_class, value> const value_ = "value";
    x3::rule<struct o_tuple_class, std::vector<value> > o_tuple_ = "tuple";

    x3::real_parser<float, x3::strict_real_policies<float> > float_;

    const auto o_tuple__def = "tuple" >> x3::lit(':') >> ("[" >> value_ % "," >> "]");

    const auto value__def
        = "float" >> (':' >> float_)
        | "int" >> (':' >> x3::int_)
        | o_tuple_
        ;

    BOOST_SPIRIT_DEFINE(value_, o_tuple_)

    const auto entry_point = x3::skip(x3::space) [ value_ ];
}

int main()
{
    for (std::string const str : {
            "float: 3.14",
            "int: 3",
            "tuple: [float: 3.14,int: 3]",
            "tuple: [float: 3.14,int: 3,tuple: [float: 4.14,int: 4]]"
    }) {
        std::cout << "============ '" << str << "'\n";

        //using boost::spirit::x3::parse;
        auto first = str.begin(), last = str.end();
        value val;

        if (parse(first, last, parser::entry_point, val))
            std::cout << "Parsed '" << val << "'\n";
        else
            std::cout << "Parse failed\n";

        if (first != last)
            std::cout << "Remaining input: '" << std::string(first, last) << "'\n";
    }
}

However I would like to use a traditional visitor class rather than making ostream a friend in the variant class. You know just a struct/class with a bunch of function objects for each type you encounter in the variant and a "for loop" for the vector that calls std::visit for each element.

My goal for the traditional visitor class is to be able to maintain a state machine during printing.

My own attempts to write this visitor class did fail because I ran into an issue with my GCC 8.1 compiler. With GCC during compilation std::variant happens to be std::variant_size somehow and I got the following error:

error: incomplete type 'std::variant_size' used in nested name specifier

More about this here: Using std::visit on a class inheriting from std::variant - libstdc++ vs libc++

Is it possible giving this constraint on GCC to write a visitor class for the code example I included, so that the ostream stuff can be removed?

Answer 1

Is it possible giving this constraint on GCC to write a visitor class for the code example I included, so that the ostream stuff can be removed?

Sure. Basically, I see three approaches:

1. Add the template machinery

You can specialize the implementation details accidentally required by GCC:

struct value: std::variant<int,float,std::vector<value>> {
    using base_type = std::variant<int,float,std::vector<value>>;
    using base_type::variant;
};

namespace std {
    template <> struct variant_size<value> :
std::variant_size<value::base_type> {};
    template <size_t I> struct variant_alternative<I, value> :
std::variant_alternative<I, value::base_type> {};
}

See it live on Wandbox (GCC 8.1)

2. Don't (again live)

Extending the std namespace is fraught (though I think it's legal for user-defined types). So, you can employ my favorite pattern and hide th estd::visit dispatch in the function object itself:

template <typename... El>
    void operator()(std::variant<El...> const& v) const { std::visit(*this, v); }

Now you can simply call the functor and it will automatically dispatch on your own variant-derived type because that operator() overload does NOT have the problems that GCC stdlib has:

    if (parse(first, last, parser::entry_point, val))
    {
        display_visitor display { std::cout };

        std::cout << "Parsed '";
        display(val);
        std::cout << "'\n";
    }

3. Make things explicit

I like this the least, but it does have merit: there's no magic and no tricks:

struct value: std::variant<int,float,std::vector<value>> {
    using base_type = std::variant<int,float,std::vector<value>>;
    using base_type::variant;

    base_type const& as_variant() const { return *this; }
    base_type&       as_variant() { return *this; }
};

struct display_visitor {
    void operator()(value const& v) const { std::visit(*this, v.as_variant()); }
     // ...

Again, live

SUMMARY

After thinking a bit more, I'd recommend the last approach, due to the relative simplicity. Clever is often a code-smell :)

Full listing for future visitors:

//#define BOOST_SPIRIT_X3_DEBUG
#include <iostream>
#include <boost/fusion/adapted.hpp>
#include <boost/spirit/home/x3.hpp>
#include <string>
#include <vector>
#include <variant>

struct value: std::variant<int,float,std::vector<value>> { 
    using base_type = std::variant<int,float,std::vector<value>>;
    using base_type::variant;

    base_type const& as_variant() const { return *this; }
    base_type&       as_variant() { return *this; }
};

struct display_visitor {
    std::ostream& _os;
    void operator()(value const& v) const { std::visit(*this, v.as_variant()); }
    void operator()(float const& f) const { _os << "float:" << f; }
    void operator()(int const& i)   const { _os << "int:" << i; }
    void operator()(std::vector<value> const& v) const { 
        _os << "tuple: [";
        for (auto& el : v) {
            operator()(el);
            _os << ",";
        }
        _os << ']';
    }
};

namespace parser {
    namespace x3 = boost::spirit::x3;

    x3::rule<struct value_class, value> const value_ = "value";
    x3::rule<struct o_tuple_class, std::vector<value> > o_tuple_ = "tuple";

    x3::real_parser<float, x3::strict_real_policies<float> > float_;

    const auto o_tuple__def = "tuple" >> x3::lit(':') >> ("[" >> value_ % "," >> "]");

    const auto value__def
        = "float" >> (':' >> float_)
        | "int" >> (':' >> x3::int_)
        | o_tuple_
        ;

    BOOST_SPIRIT_DEFINE(value_, o_tuple_)

    const auto entry_point = x3::skip(x3::space) [ value_ ];
}

int main()
{
    for (std::string const str : {
        "float: 3.14",
        "int: 3",
        "tuple: [float: 3.14,int: 3]",
        "tuple: [float: 3.14,int: 3,tuple: [float: 4.14,int: 4]]"
    }) {
        std::cout << "============ '" << str << "'\n";

        //using boost::spirit::x3::parse;
        auto first = str.begin(), last = str.end();
        value val;

        if (parse(first, last, parser::entry_point, val))
        {
            display_visitor display { std::cout };

            std::cout << "Parsed '";
            display(val);
            std::cout << "'\n";
        }
        else
            std::cout << "Parse failed\n";

        if (first != last)
            std::cout << "Remaining input: '" << std::string(first, last) << "'\n";
    }
}