"not resolved from current namespace" error, when calling C routines from R

Question

I am recently doing some computational testing with mgcv GAM. Some original functions are modified, and some are added. In order not to break compatibility, for every function I want to modify, I create a new version with a .zheyuan surfix in function name. For example, with Sl.fit function for doing penalized least squares fitting, I will have an Sl.fit.zheyuan. I would simply collect all R functions I write into a standalone R script "zheyuan.R". By adding this file into the R directory of mgcv_1.8-17 package source, and compiling this modified package into a local path, I could load it for testing purpose.

I have no problem in adding R routines, but not when adding C routines. No error occurs when installing the modified package, but when I call the R wrapper function of my added C routine, I would get the error as in my question title. If you are interested in my case, you may follow the following steps to reproduce such error.

Step 1: download the latest package source

Download the 1.8-17 version from the above link. Such link will die when new mgcv release is published on CRAN. But you can always go to mgcv CRAN page to download the latest release.

Let's untar the source. First, remove the file MD5, so that we don't get annoying MD5 warning when compiling the modified version. In the following, we would add new stuff into R directory, and src directory.

Step 2: create an R script

Consider the following R wrapper function:

RX <- function (R, X) {
  X <- X + 0
  .Call("C_mgcv_RX", R, X)
  X
  }

Create an R script "zheyuan.R" to place this function, and add it into mgcv/R.

Step 3: adding C routines

C routines regarding matrix computations are normally under src/mat.c. So let's append a new function to the end of this script:

void mgcv_RX (SEXP R, SEXP X) {
  int nrowX = nrows(X);
  int ncolX = ncols(X);
  double one = 1.0;
  F77_CALL(dtrmm)("l", "u", "n", "n", &nrowX, &ncolX, &one, REAL(R), &nrowX, REAL(X), &nrowX);
  }

This is a simple routine multiplying an upper triangular matrix R, with a rectangular matrix X. The output matrix will overwrite X. Level-3 BLAS dtrmm will be called for this purpose. We don't need to worry about header files or runtime linking to BLAS library. Headers are availalbe in mat.c, and linking to BLAS is managed by R.

Step 4: register C routine

The above is insufficient. Each C routine in mgcv will appear in three places. For example, let's try searching a native C routine:

grep mgcv_RPPt mgcv/src/*
# mgcv/src/init.c:  {"mgcv_RPPt",(DL_FUNC)&mgcv_RPPt,3},
# mgcv/src/mat.c:void mgcv_RPPt(SEXP a,SEXP r, SEXP NT) {
# mgcv/src/mgcv.h:void mgcv_RPPt(SEXP a,SEXP r, SEXP NT);

We also need to append a header file mgcv.h, and register this C routine in init.c.

Let's append

void mgcv_RX (SEXP R, SEXP X);

to the end of mgcv.h, and inside init.c, do:

R_CallMethodDef CallMethods[] = {
  {"mgcv_pmmult2", (DL_FUNC) &mgcv_pmmult2,5},
  {"mgcv_Rpiqr", (DL_FUNC) &mgcv_Rpiqr,5}, 
  {"mgcv_tmm",(DL_FUNC)&mgcv_tmm,5}, 
  {"mgcv_Rpbsi",(DL_FUNC)&mgcv_Rpbsi,2},
  {"mgcv_RPPt",(DL_FUNC)&mgcv_RPPt,3},
  {"mgcv_Rpchol",(DL_FUNC)&mgcv_Rpchol,4},
  {"mgcv_Rpforwardsolve",(DL_FUNC)&mgcv_Rpforwardsolve,3},
  {"mgcv_Rpcross",(DL_FUNC)&mgcv_Rpcross,3},
  {"mgcv_RX",(DL_FUNC)&mgcv_RX,2},  //  we add this line
  {NULL, NULL, 0}
};

Step 5: complie and load

tar the modified mgcv folder to mgcv.tar.gz.

Open up a new, clean R session (possibly you need R --vanilla for start-up). Then specifying a local library path and run:

path <- getwd()  ## let's just use current working directory
## make sure you move "mgcv.tar.gz" into current working path
install.packages("mgcv.tar.gz", repos = NULL, lib = path)
library(mgcv, lib.loc = path)

Step 6: test and get error

R <- matrix(runif(25), 5)
R[lower.tri(R)] <- 0
X <- matrix(runif(25), 5)
mgcv:::RX(R, X)  ## function is not exported, so use `mgcv:::` to find it

# Error in .Call("C_mgcv_RX", R, X) : 
#   "C_mgcv_RX" not resolved from current namespace (mgcv)

Could anyone explain why and how to resolve this?

Answer 1

I have a temporary "fix" now. Instead of

.Call("C_mgcv_RX", R, X)

use either of the following:

.Call(mgcv:::"C_mgcv_RX", R, X)
.Call(getNativeSymbolInfo("mgcv_RX"), R, X)

I came about this because I suddenly realize that C routines can be extracted by :::, too. Since package compilation is successful, there is no way that mgcv::: can not find this C routine. And yes, it works.

---

To check that our defined C routine is available in the shared library loaded, try

is.loaded("mgcv_RX")
# TRUE

To list all registered C routines in the loaded shared library, use

getDLLRegisteredRoutines("mgcv")