How to do a generic reusable rules in Boost::Spirit::x3?

Question

In a parser I am currently working on, I am writing some rules/"sub-parser" whose size is rather significant. The thing is: as this parser can demand a certain effort from the compiler and can have a certain size, I would like to translate and instantiate it only once and thereby write the rule like explained in the boost tutorial. So far, so good. BUT, this rule will be meant to be reused frequently by different parsers whose attribute type is always different.

I will illustrate my problem with the trivial example below which is already something a tried and that won't build. my_rule is the rule that has to be reusable. I want it to be usable in three parsing rules for Foo, Bar and Baz. They each represent possible use cases of re-usability I could encounter. So far, as shown in the example, I tried to use a boost::fusion::vector a generic attribute for all of the use cases but it ain't that easy. The parsing rules for Foo and Baz build but the one for Bar does not.

Thus, my question is, in the light of this example: How to make my_rule work in every use case ? What to use as an attribute ? Would it be at least possible ? I would be ready to hear "no".

#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/fusion/include/vector.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/binary.hpp>

namespace x3 = boost::spirit::x3;

struct Foo
{
  int a;
  int b;
};

struct Bar
{
  int c;
  int d;
  int e;
};

struct Baz
{
  int f;
  Foo foo;
};

BOOST_FUSION_ADAPT_STRUCT(Foo, a, b);
BOOST_FUSION_ADAPT_STRUCT(Bar, c, d, e);
BOOST_FUSION_ADAPT_STRUCT(Baz, f, foo);

const auto my_rule =
  x3::rule<class My_rule_tag, boost::fusion::vector<int, int>>{ "my-rule" } %=
  x3::byte_ >> x3::byte_;

const auto foo_rule = x3::rule<class Foo_rule_tag, Foo>{ "foo-rule" } %=
  my_rule;

const auto bar_rule = x3::rule<class Bar_rule_tag, Bar>{ "bar-rule" } %=
  x3::byte_ >> my_rule;

const auto baz_rule = x3::rule<class Bar_rule_tag, Baz>{ "baz-rule" } %=
  x3::byte_ >> my_rule;

int
main()
{
  std::vector<std::uint8_t> frame{ 0x03, 0x06, 0x09 };
  // Build OK
  Foo foo1;
  x3::parse(frame.begin(), frame.end(), foo_rule, foo1);
  // Build NOK
  Bar bar1;
  x3::parse(frame.begin(), frame.end(), bar_rule, bar1);
  // Build OK
  Baz baz1;
  x3::parse(frame.begin(), frame.end(), baz_rule, baz1);
  return 0;
}

Answer 1

To be honest the goal seems conflicted. For generic and reusable you want the rules to be in the same TU as the invocation.

As you've already figured out (see Baz aggregating Foo) the key is to make the attribute types compatible for attribute propagation.

The simplest thing I can think of:

Live On Coliru

#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/binary.hpp>

template <typename T> constexpr static T parse(auto const& frame, auto rule) {
    T val;
    parse(begin(frame), end(frame), rule, val);
    return val;
}

struct Foo { int a, b;       constexpr auto operator<=>(Foo const&) const = default; };
struct Bar { int c, d, e;    constexpr auto operator<=>(Bar const&) const = default; };
struct Baz { int f; Foo foo; constexpr auto operator<=>(Baz const&) const = default; };
BOOST_FUSION_ADAPT_STRUCT(Foo, a, b)
BOOST_FUSION_ADAPT_STRUCT(Bar, c, d, e)
BOOST_FUSION_ADAPT_STRUCT(Baz, f, foo.a, foo.b)

// using Vec2 = boost::fusion::vector<int, int>;
namespace x3  = boost::spirit::x3;

auto const vec2_    = x3::byte_ >> x3::byte_;
auto const foo_rule = vec2_;
auto const bar_rule = x3::byte_ >> vec2_;
auto const baz_rule = x3::byte_ >> foo_rule;

int main() {
    constexpr std::array<std::uint8_t, 3> frame{0x03, 0x06, 0x09};
    assert((parse<Foo>(frame, foo_rule) == Foo{ 3, 6 }));
    assert((parse<Bar>(frame, bar_rule) == Bar{ 3, 6, 9 }));
    assert((parse<Baz>(frame, baz_rule) == Baz{ 3, {6, 9} }));
}

note the adaptation of Baz!

If you want {foo,bar,baz}_rule properly compilation-firewalled:

Live On Coliru

#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/binary.hpp>

template <typename T> constexpr static T parse(auto const& frame, auto rule) {
    T val;
    parse(begin(frame), end(frame), rule, val);
    return val;
}

struct Foo { int a, b;       constexpr auto operator<=>(Foo const&) const = default; };
struct Bar { int c, d, e;    constexpr auto operator<=>(Bar const&) const = default; };
struct Baz { int f; Foo foo; constexpr auto operator<=>(Baz const&) const = default; };
BOOST_FUSION_ADAPT_STRUCT(Foo, a, b)
BOOST_FUSION_ADAPT_STRUCT(Bar, c, d, e)
BOOST_FUSION_ADAPT_STRUCT(Baz, f, foo)

// using Vec2 = boost::fusion::vector<int, int>;
namespace x3  = boost::spirit::x3;

auto const vec2_    = x3::byte_ >> x3::byte_;
auto const foo_rule = x3::rule<class Foo_rule_tag, Foo>{"foo_rule"} = vec2_;
auto const bar_rule = x3::rule<class Bar_rule_tag, Bar>{"bar_rule"} = x3::byte_ >> vec2_;
auto const baz_rule = x3::rule<class Baz_rule_tag, Baz>{"baz_rule"} = x3::byte_ >> foo_rule;

int main() {
    constexpr std::array<std::uint8_t, 3> frame{0x03, 0x06, 0x09};
    assert((parse<Foo>(frame, foo_rule) == Foo{ 3, 6 }));
    assert((parse<Bar>(frame, bar_rule) == Bar{ 3, 6, 9 }));
    assert((parse<Baz>(frame, baz_rule) == Baz{ 3, {6, 9} }));
}

If vec2_ (probably a packet header of sorts) is "involved" then

• either aggregate it just like you did in Baz
• include it's generic definition in each TU that needs it

To be completely honest, seperating your parser across TUs is never worth it, especially for X3. The compile times are already very low, and the explicit instantiations hard-lock the iterator as well as the context types, often leading to inscrutable linker errors.

See e.g.

• Mixing non-terminal rules from separeted translation unit
• BOOST_SPIRIT_DEFINE not understand

OUT OF THE BOX

Since it looks like you're actually binary-serializing packed structs (with some application specific wire representation), consider generating your parsers and serializers e.g. using Boost Describe.