C++11 lambdas and parameter packs

Question

I'm having essentially the same problem as this question, however unfortunately the only posted answer there is now a dead link.

Specifically, using VS2013 Update 4, I'm trying to get the following code to compile, and it's not being cooperative:

template<typename T, typename... Params>
auto PostWeakTask(const boost::weak_ptr<IWorkQueue>& queue,
                  void (T::*member)(Params...),
                  const boost::weak_ptr<T>& weak)
    -> std::function<void (Params&&...)>
{
    return [queue, member, weak](Params&&... params)
    {
        if (auto qp = queue.lock())
        {
            qp->Post([weak, member, params]()
            {
                if (auto strong = weak.lock())
                {
                    ((*strong).*member)(std::forward<Params>(params)...);
                }
            });
        }
    };
}

(As written, the capture of params fails with C3481: 'params': lambda capture variable not found. If I try using implicit capture with = instead, it says C2065: 'params' : undeclared identifier. If I try params..., I get C3521: 'params' is not a parameter pack.)

The idea of course is to return a function that when called will post a member function with arbitrary parameters to a work queue that accepts void() tasks only, and to only keep weak references to the queue and task when not yet executed.

I don't think there's anything actually wrong in the code here, though. I think I've found a workaround using bind instead of a lambda, but oddly it only seems to work with std::function and not boost::function. (Using Boost 1.55. I suspect this might be pre-rvalue-ref support?)

(On a side note, I originally tried to use decltype([](Params&&...){}) as the return type, as this seems more natural. But it crashes the compiler.)

---

The workaround is doing something a bit weird too. Maybe I should ask this as a separate question as it mostly relates to the perfect-forwarding part instead, but:

template<typename T, typename... Params>
auto PostWeakTask(const boost::weak_ptr<IWorkQueue>& queue,
                  void (T::*member)(Params...),
                  const boost::weak_ptr<T>& weak)
    -> std::function<void (Params...)>
{
    struct WeakCaller
    {
        typedef void (T::*member_type)(Params...);
        typedef boost::weak_ptr<T> weak_type;

        WeakCaller(member_type member, const weak_type& weak)
            : m_member(member), m_weak(weak) {}

        void operator()(Params&&... params)
        {
            if (auto strong = m_weak.lock())
            {
                ((*strong).*m_member)(std::forward<Params>(params)...);
            }
        }

    private:
        member_type m_member;
        weak_type m_weak;
    };

    return [=](Params&&... params)
    {
        if (auto qp = queue.lock())
        {
            qp->Post(std::bind(WeakCaller(member, weak),
                     std::forward<Params>(params)...));
        }
    }
}

This seems like it should work, but when I try calling it (with a void (tribool,tribool,const std::string&) method) I get a binding error that a tribool parameter is not compatible with a tribool&& one.

(Specifically: C2664: 'void WeakCaller<T,boost::logic::tribool,boost::logic::tribool,const std::string &>::operator ()(boost::logic::tribool &&,boost::logic::tribool &&,const std::string &)' : cannot convert argument 1 from 'boost::logic::tribool' to 'boost::logic::tribool &&'.)

I thought part of the point of using rvalue references this way was that they were supposed to forward perfectly without needing multiple overloads?

I can make it compile by making WeakCaller have void operator()(Params... params) instead, but doesn't this defeat perfect forwarding? (Oddly it seems ok with leaving the && at the top-level lambda... and I'm not sure if the mismatch between the std::function signature and the lambda signature is ok.)

---

Just for reference, here is the final version I'm using now, after tweaking Oktalist's answer:

namespace detail
{
    template<size_t... Ints> struct index_sequence
    {
        static size_t size() { return sizeof...(Ints); }
    };

    template<size_t Start, typename Indices, size_t End>
    struct make_index_sequence_impl;

    template<size_t Start, size_t... Indices, size_t End>
    struct make_index_sequence_impl<Start, index_sequence<Indices...>, End>
    {
        typedef typename make_index_sequence_impl<
            Start + 1, index_sequence<Indices..., Start>, End>::type type;
    };

    template<size_t End, size_t... Indices>
    struct make_index_sequence_impl<End, index_sequence<Indices...>, End>
    {
        typedef index_sequence<Indices...> type;
    };

    template <size_t N>
    using make_index_sequence = typename
        make_index_sequence_impl<0, index_sequence<>, N>::type;

    template<typename... Ts>
    using index_sequence_for = make_index_sequence<sizeof...(Ts)>;

    template<typename... Ts>
    struct MoveTupleWrapper
    {
        MoveTupleWrapper(std::tuple<Ts...>&& tuple)
            : m_tuple(std::move(tuple)) {}
        MoveTupleWrapper(const MoveTupleWrapper& other)
            : m_tuple(std::move(other.m_tuple)) {}
        MoveTupleWrapper& operator=(const MoveTupleWrapper& other)
            { m_tuple = std::move(other.m_tuple); }

        template<typename T, typename... Params>
        void apply(void (T::*member)(Params...), T& object) const
        {
            applyHelper(member, object, index_sequence_for<Ts...>());
        }

        template<typename T, typename... Params, size_t... Is>
        void applyHelper(void (T::*member)(Params...), T& object,
                         index_sequence<Is...>) const
        {
            (object.*member)(std::move(std::get<Is>(m_tuple))...);
        }

    private:
        mutable std::tuple<Ts...> m_tuple;
    };

    template<typename... Ts>
    auto MoveTuple(Ts&&... objects)
        -> MoveTupleWrapper<std::decay_t<Ts>...>
    {
        return std::make_tuple(std::forward<Ts>(objects)...);
    }

    template<typename T, typename... Params>
    struct WeakTaskPoster
    {
        WeakTaskPoster(const boost::weak_ptr<IWorkQueue>& queue,
                       void (T::*member)(Params...),
                       const boost::weak_ptr<T>& weak) :
            m_queue(queue),
            m_member(member),
            m_weak(weak)
        {}

        template<typename... XParams>
        void operator()(XParams&&... params) const
        {
            if (auto qp = m_queue.lock())
            {
                auto weak = m_weak;
                auto member = m_member;
                auto tuple = MoveTuple(std::forward<XParams>(params)...);
                qp->Post([weak, member, tuple]()
                {
                    if (auto strong = weak.lock())
                    {
                        tuple.apply(member, *strong);
                    }
                });
            }
        }

    private:
        boost::weak_ptr<IWorkQueue> m_queue;
        void (T::*m_member)(Params...);
        boost::weak_ptr<T> m_weak;
    };
}

template<typename T, typename... Params>
auto PostWeakTask(const boost::weak_ptr<IWorkQueue>& queue,
                  void (T::*member)(Params...),
                  const boost::weak_ptr<T>& weak)
    -> detail::WeakTaskPoster<T, Params...>
{
    return { queue, member, weak };
}

I guess it's still not truly universal since it doesn't work with lvalue references, but since those are dangerous in a delayed callback context anyway this is not that important.

(Although, if you're really desperate to pass lvalue references around, it appears to work if you call with std::ref(x) and receive as std::reference_wrapper<T> (or const& or &&). There's probably some additional magic that could make this transparent, but I don't really need it so I haven't investigated.)

Answer 1

I thought part of the point of using rvalue references this way was that they were supposed to forward perfectly without needing multiple overloads?

No, rvalue references and forwarding references are two different things. They both involve && but they behave differently. Forwarding references occur only in the presence of template argument deduction (or auto type deduction, which uses the same rules).

This is a forwarding reference:

template<typename T>
void foo(T&& v);

Template argument deduction will let T be any type; if T is an lvalue reference then v is an lvalue reference, otherwise v is an rvalue reference, due to reference collapsing.

This is NOT a forwarding reference:

void foo(int&& v);

There is no template argument deduction here, so v is a plain rvalue reference to int, it can bind only to rvalues.

This is also NOT a forwarding reference:

template<typename T>
struct Foo
{
    void bar(T&& v);
};

Again because there is no template argument deduction. The function Foo<int>::bar is not a template, it is a plain function taking a plain rvalue reference parameter which can bind only to rvalues. The function Foo<int&>::bar is a plain function taking a plain lvalue reference parameter which can bind only to lvalues.

On a side note, I originally tried to use decltype([](Params&&...){}) as the return type, as this seems more natural.

That would never work. Every lambda expression denotes a unique type, so decltype([]{}) and decltype([]{}) are two different, unrelated types.

Here is a possible solution. The only C++14 feature used is std::index_sequence, for which you can Google a C++11 implementation.

template<typename... Ts>
struct MoveTupleWrapper
{
    MoveTupleWrapper(std::tuple<Ts...>&& tuple) : m_tuple(tuple) {}
    MoveTupleWrapper(MoveTupleWrapper& other) : m_tuple(std::move(other.m_tuple)) {}
    MoveTupleWrapper& operator=(MoveTupleWrapper& other) { m_tuple = std::move(other.m_tuple); }
    std::tuple<Ts...> m_tuple;

    template<typename T, typename... Params>
    void apply(void (T::*member)(Params...), T& object)
    {
        applyHelper(member, object, std::index_sequence_for<Ts...>);
    }
    template<typename T, typename... Params, size_t... Is>
    void applyHelper(void (T::*member)(Params...), T& object, std::index_sequence<Is...>)
    {
        (object.*member)(std::move(std::get<Is>(m_tuple))...);
    }
};

template<typename... Ts>
auto MoveTuple(Ts&&... objects)
    -> MoveTupleWrapper<std::decay_t<Ts>...>
{
    return std::make_tuple(std::forward<Ts>(objects)...);
}

template<typename T, typename... Params>
struct WeakTaskPoster
{
    WeakTaskPoster(const boost::weak_ptr<IWorkQueue>& queue,
                   void (T::*member)(Params...),
                   const boost::weak_ptr<T>& weak) :
        m_queue(queue),
        m_member(member),
        m_weak(weak)
    {}
    boost::weak_ptr<IWorkQueue> m_queue;
    void (T::*m_member)(Params...);
    boost::weak_ptr<T> m_weak;

    template<typename... XParams>
    void operator()(XParams&&... params) const
    {
        if (auto qp = m_queue.lock())
        {
            auto weak = m_weak;
            auto member = m_member;
            auto tuple = MoveTuple(std::forward<XParams>(params)...);
            qp->Post([weak, member, tuple]() mutable
            {
                if (auto strong = weak.lock())
                {
                    tuple.apply(member, *strong);
                }
            });
        }
    }
};

template<typename T, typename... Params>
auto PostWeakTask(const boost::weak_ptr<IWorkQueue>& queue,
                  void (T::*member)(Params...),
                  const boost::weak_ptr<T>& weak)
    -> WeakTaskPoster<T, Params...>
{
    return { queue, member, weak };
}

Note that WeakTaskPoster::operator() is a template, so we obtain perfect forwarding. The C++14 way to do that would've been a generic lambda. Then there is this MoveTuple thing. That's a wrapper around a std::tuple which simply implements copy in terms of move, so we can work around the lack of C++14 lambda capture-by-move. It also implements exactly the subset of std::apply that is necessary.

It's very similar to Yakk's answer, except that the tuple of parameters is moved instead of copied into the lambda's capture set.

Answer 2

Yakk's answer did inspire me a bit to make a slight alteration to the workaround posted above:

template<typename T, typename... Params>
struct WeakCaller
{
    typedef void (T::*member_type)(Params...);
    typedef boost::weak_ptr<T> weak_type;

    WeakCaller(member_type member, const weak_type& weak)
        : m_member(member), m_weak(weak) {}

    template<typename... Args>
    void operator()(Args&&... params)
    {
        if (auto strong = m_weak.lock())
        {
            ((*strong).*m_member)(std::forward<Args>(params)...);
        }
    }

private:
    member_type m_member;
    weak_type m_weak;
};

template<typename T, typename... Params>
auto PostWeakTask(const boost::weak_ptr<IWorkQueue>& queue,
                  void (T::*member)(Params...),
                  const boost::weak_ptr<T>& weak)
    -> std::function<void (Params...)>
{
    return [=](Params&&... params)
    {
        if (auto qp = queue.lock())
        {
            qp->Post(std::bind(WeakCaller<T, Params...>(member, weak),
                     std::forward<Params>(params)...));
        }
    }
}

This appears to compile and work as expected, although I haven't tested it with move-only types yet (I suspect they will be problematic due to the use of bind and function). I'm not really sure why re-templating the operator was needed, but it seems to bypass the compiler errors I was having with && params.

I'm also still not sure whether the return type of PostWeakTask should be std::function<void(Params...)> or std::function<void(Params&&...)>; it seems to compile ok both ways.

Answer 3

Make things work is my goal.

I'll attack this:

qp->Post([weak, member, params]()
{
  if (auto strong = weak.lock()) {
    ((*strong).*member)(std::forward<Params>(params)...);
  }
});

First, let us get params into the lambda.

auto tParams = std::make_tuple( std::forward<Params>(params)... );
qp->Post([weak, member, tParams]()
{
  if (auto strong = weak.lock()) {
    ((*strong).*member)(std::forward<Params>(params)...);
  }
});

however, you'll note that now the body doesn't compile!

We are, however, closer.

template<class T, class M>
struct member_invoke_t;
template<class T, class R, class...Args>
struct member_invoke_t<T*, R(::*)(Args...)> {
  T* t;
  R(::*m)(Args...);
  template<class...Ts>
  R operator()(Ts&&...ts)const{
    return (t->*m)(std::forward<Ts>(ts)...);
  }
};
template<class T, class M>
member_invoke_t<T*, M*> member_invoke(T* t, M*m) {
  return {t, m};
};

then we only have to write std::apply

qp->Post([weak, member, tParams]()mutable
{
  if (auto strong = weak.lock())
    std::apply( member_invoke(strong.get(), member), std::move(tParams) );
}

which should be doable. (The above code assumes that the lambda passed to Post will only be invoked once by the way -- hence the move and the mutable).

std::apply has a possible C++14 implementation over here. Converting to C++11 requires writing index_sequence and replacing the decltype(auto) with an explicit ->decltype(expression) I think. In any case, that is a sub-problem I'll skip here.