modifying custom type Rcpp::List objects in place

Question

I am struggling to figure out how to modify custom type list elements in an Rcpp::List; below is some code to illustrate my problem (using Rcpp modules).

#include <Rcpp.h>

class Base {

public:

    Rcpp::NumericVector data;

    Base() {
        this->data = Rcpp::NumericVector();
    }

    void append(double x) {
        data.push_back(x);
    }

};
RCPP_EXPOSED_CLASS(Base)


class Container {

public:

    Rcpp::List objects;

    Container(Rcpp::List objects) {
        this->objects = objects;
    }

    void append(double x) {
        for (int i = 0; i < objects.length(); i++) {
            // cannot modify in-place because objects[i] is a temporary!
            const Base *obj = objects[i];
            // try to outsmart the compiler by copying to non const - lvalue hell!
            Base *ptr = obj;
            ptr->append(x);
        }
    }

    Base* at(int i) {
        void *ptr = objects[i];
        return (Base*) ptr;
    }

};
RCPP_EXPOSED_CLASS(Container) 

The problem is that I need to call the method Base::append and thus need to get a non-const pointer to the i-th list element. Since objects[i] is a temporary object, I cannot define a non-const pointer though. Here I tried copying the const pointer, but the compiler complains about not being able to initialize the non-const pointer with a const lvalue (I guess that means it realized I was trying to outsmart it).

Do I need to use another (typed) collection instead of Rcpp::List or how can I make this work?

Answer 1

You need to understand how R objects work (Rcpp being a wrapper). The key is

1. R objects are all pointers (SEXP being a pointer to SEXPREC)

2. When you push_back to a vector you are potentially changing the underlying pointer.

Therefore your NumericVectors are not guaranteed to refer to the same object after push_back.

One option is to build your Base class using references. Here's an example.

sourceCpp("mycontainer.cpp")
x <- as.list(1:5)
mc <- new(Container, x)
mc$append(6)
print(x)
[[1]]
[1] 1 6

[[2]]
[1] 2 6

[[3]]
[1] 3 6
...

Rcpp code:

#include <Rcpp.h>

class Base {
public:
  Rcpp::NumericVector & data;
  Base(Rcpp::NumericVector & x) : data(x) {}
  void append(double x) {
    data.push_back(x);
  }
};
RCPP_EXPOSED_CLASS(Base)
  
  
class Container {
  
public:
  Rcpp::List objects;
  Container(Rcpp::List objects) {
    this->objects = objects;
  }
  void append(double x) {
    for (int i = 0; i < objects.length(); i++) {
      Rcpp::NumericVector objvec = objects[i];
      Base obj( objvec );
      obj.append(x);
      objects[i] = objvec;
    }
  }
  Rcpp::List output() {
    return objects;
  }
};
RCPP_EXPOSED_CLASS(Container)

RCPP_MODULE(Container){
  using namespace Rcpp;
  class_<Container>("Container")
  .constructor< List >()
  .method( "append", &Container::append )
  .method( "output", &Container::output )
  ;
}

Edit: You can see how the pointer (SEXP) changes on push_back.

// [[Rcpp::export]]
void test() {
  IntegerVector x(0);
  std::cout << (void*) x.get__() << std::endl;
  x[0] = 50;
  std::cout << (void*) x.get__() << std::endl; // no change
  x.push_back(1);
  std::cout << (void*) x.get__() << std::endl; // address change
}

Answer 2

I don't really have time to work through your problem but from a first glance it appears to be related to the objects instance being local to your object which runs into issue that have more to do with C++ lifetime scope than with Rcpp interfaces.

When I wanted or needed more permanent "collections" of things, I usually stripped the problem away by ensuring the "collector", here your objects was global and lasted. One (very basic) solution for that is a static pointer which a few helper functions to a) initialize, b) add an object, c) fetch an object (by key or position) and of course d) unwind / cleanup at end. It is still a little tricky because you want to make sure there aren't multiple copies of your collector (the static part helps).