Where do absent dots (`...`) get processed?

Question

If we view the body of a function that has dots ... in its argument list, we can usually find the function that receives those dots arguments.

For example, we can see in the body of sapply() that the dots arguments are passed through to lapply().

sapply
# function (X, FUN, ..., simplify = TRUE, USE.NAMES = TRUE) 
# {
#     FUN <- match.fun(FUN)
#     answer <- lapply(X = X, FUN = FUN, ...)
#     ## rest of function body
# }
# <bytecode: 0x000000000e05f0b0>
# environment: namespace:base>

However, in lapply(), there are dots ... in the argument list but not in the function body.

lapply
# function (X, FUN, ...) 
# {
#     FUN <- match.fun(FUN)
#     if (!is.vector(X) || is.object(X)) 
#         X <- as.list(X)
#     .Internal(lapply(X, FUN))
# }
# <bytecode: 0x0000000009414f08>
# <environment: namespace:base>

So where do the dots ... arguments in lapply() get processed? What/where are they passed to? We cannot pass them to match.fun(). I presume they are passed into .Internal() but I don't see any reason for this to work when I don't see them passed into any function in the function body.

Answer 1

However, in lapply(), there are dots ... in the argument list but not in the function body.

But they are in the environment in the body of the lapply! The lines of interest are:

• the do_apply C function.
• this ICONS call.
• this R_forceAndCall.

We can do something very similar with the C++ file below which we can Rcpp::sourceCpp where we use Rf_lang3 instead of the calling ICONS:

 #include <Rcpp.h>

// [[Rcpp::export(rng = false)]]
SEXP to_simple_call_f_1(SEXP f, SEXP x, SEXP env){
  SEXP fn  = PROTECT(Rf_lang3(f, x, R_DotsSymbol)), 
       ans = Rf_eval(fn, env);
  UNPROTECT(1);
  return ans;
}

// [[Rcpp::export(rng = false)]]
SEXP to_simple_call_f_2(SEXP f, SEXP x, SEXP rho){
  SEXP fn  = PROTECT(Rf_lang3(f, x, R_DotsSymbol)), 
       ans = R_forceAndCall(fn, 1, rho);
  UNPROTECT(1);
  return ans;
}

// [[Rcpp::export(rng = false)]]
SEXP to_simple_call_f_3(SEXP f, SEXP x, SEXP rho){
  SEXP fn  = PROTECT(Rf_lang2(f, x)), 
       ans = Rf_eval(fn, rho);
  UNPROTECT(1);
  return ans;
}

We can now see that:

f <- function(x, ...)
  x * unlist(list(...))
w_wont_work <- function(x, ...)
  to_simple_call_f_1(f = f, x = x, .GlobalEnv)

# first the solution that does not work (wrong env!)
a <- 3
b <- 2
w_wont_work(x, a, b)
#R> Error in to_simple_call_f_1(f = f, x = x, .GlobalEnv) : 
#R>   '...' used in an incorrect context

# now with the correct versions
w_1 <- function(x, ...)
  to_simple_call_f_1(f = f, x = x, environment())
w_2 <- function(x, ...)
  to_simple_call_f_2(f = f, x = x, environment())
# version that uses variables from the global environment instead
f3 <- function(x)
  x * unlist(list(a, b))
w_3 <- function(x)
  to_simple_call_f_3(f = f3, x = x, environment())

# check the functions
f  (2, a, b)
#R> [1] 6 4
w_1(2, a, b)
#R> [1] 6 4
w_2(2, a, b)
#R> [1] 6 4
w_3(2)
#R> [1] 6 4

# almost runs in the same time
bench::mark(f(2, a, b), w_1(2, a, b), w_2(2, a, b), w_3(2), min_time = 1, 
            max_iterations = 1e9)
#R> # A tibble: 4 x 13
#R>   expression        min  median `itr/sec` mem_alloc `gc/sec`  n_itr  n_gc total_time 
#R>   <bch:expr>   <bch:tm> <bch:t>     <dbl> <bch:byt>    <dbl>  <int> <dbl>   <bch:tm> 
#R> 1 f(2, a, b)      995ns  1.15µs   801375.        0B     25.1 733096    23      915ms 
#R> 2 w_1(2, a, b)   1.96µs  2.25µs   420623.    6.77KB     26.0 388953    24      925ms 
#R> 3 w_2(2, a, b)   1.93µs  2.21µs   424811.    6.77KB     25.9 393580    24      926ms 
#R> 4 w_3(2)         1.76µs  1.99µs   474501.   15.23KB     22.5 442139    21      932ms

I presume they are passed into .Internal() but I don't see any reason for this to work when I don't see them passed into any function in the function body.

Notice that the functions from Rcpp use .Call and not .Internal:

to_simple_call_f_1
#R> function (f, x, env) 
#R> .Call(<pointer: 0x7fedc44cd1b0>, f, x, env)

Thus, I pass the environment. So it seems that .Internal passes an environment I guess? ?.Internal is not that helpful.

Answer 2

They're not explicitly passed to .Internal, but I believe they're available to do_lapply (in src/main/apply.c) via dynamic scope. The scoping rules may be slightly different than usual, since .Internal is a primitive function.

You can see that ... (R_DotsSymbol) is added to the function call lapply creates, so they're available to the function call on each list element. tmp is roughly equivalent to X[[i]] and R_fcall is roughly equivalent to FUN(X[[i]], ...).

SEXP tmp = PROTECT(LCONS(R_Bracket2Symbol,
        LCONS(X, LCONS(isym, R_NilValue))));
SEXP R_fcall = PROTECT(LCONS(FUN,
             LCONS(tmp, LCONS(R_DotsSymbol, R_NilValue))));