Reputation: 2769
`r_make()` and `make()` not consistent in r-drake
I'm using RStudio for the work in question. I used to use drake::make() ignoring the prompt to use r_make() till yesterday when I decided to give it a try.
Now, I'm in a bit of a pickle. Not sure what I've done / if I have found a bug.
My project plan is not large, but the target builds are large (10-20 min data pulls each). I've run it end-to-end using make() from the RStudio console. Everything is up-to-date. Even if I open a separate R session in terminal, and run outdated(), I don't see anything outdated.
However, running r_make() from the Rstudio console, or from R in a separate terminal results in many targets out of date.
And no matter how many times I run r_make(), these objects remain un-updated.
I don't know how to proceed. I'd like to keep using r_make(), but I'm stuck.
Update
Reproducible example at https://gitlab.com/rsangole/drake-test
Update 2
Running @landau's reprex example, output is here. It has worked correctly, as Will obtained with this script: https://gitlab.com/rsangole/drake-test/snippets/1894934
However...
If I run https://gitlab.com/rsangole/drake-test/blob/master/make.R in interactive mode in RStudio, I can reproduce my condition. To reproduce:
- Open new Rstudio, clean env
- Download reproducible example in https://gitlab.com/rsangole/drake-test
- Run https://gitlab.com/rsangole/drake-test/blob/master/make.R
- Expect output as in https://gitlab.com/rsangole/drake-test/snippets/1894937
Same issue even after updating drake to latest bleeding edge version
Also able to replicate issue if I run make.R via R in zsh thereby bypassing RStudio completely.
Upvotes: 2
Views: 275
Answers (2)
Reputation: 2769
I found the issue. I was using an MRAN snapshot of '2019-06-01'-ish. Upgrading all packages to MRAN of Oct-10 fixed everything.
Upvotes: 0
Reputation: 5841
I created a reprex using https://gitlab.com/rsangole/drake-test, and I could not reproduce what you are seeing. Are you using drake differently than in the code below?
library(drake)
# Write script files
package_code <- quote({
library(data.table)
library(drake)
})
function_code <- quote(
make_data <- function(download_date){
data.table(
dt = download_date,
x = runif(n = 1e5),
y = sample(letters,size = 1e5,replace = T)
)
}
)
plan_code <- quote(
plan <- drake_plan(
download_date = "2019-09-08",
x = make_data(download_date)
)
)
writeLines(deparse(package_code), "packages.R")
writeLines(deparse(function_code), "functions.R")
writeLines(deparse(plan_code), "plan.R")
# Run make()
source("packages.R")
source("functions.R")
source("plan.R")
make(
plan,
verbose = 2,
parallelism = "future",
jobs = 4L,
lock_envir = FALSE
)
#>
target download_date
#>
target x
make(
plan,
verbose = 2,
parallelism = "future",
jobs = 4L,
lock_envir = FALSE
)
#>
All targets are already up to date.
config <- drake_config(plan)
outdated(config)
#> character(0)
# Write _drake.R
drake_code <- quote({
source("packages.R")
source("functions.R")
source("plan.R")
drake_config(
plan,
verbose = 2,
jobs = 3L,
lock_envir = FALSE,
parallelism = "future"
)
})
writeLines(deparse(drake_code), "_drake.R")
# Run r_make()
r_make()
#>
#> ..
[34mAll targets are already up to date.[39m
r_make()
#>
#> .
[34mAll targets are already up to date.[39m
r_outdated()
#>
#> .
#> character(0)
Created on 2019-09-13 by the reprex package (v0.3.0)
devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#> setting value
#> version R version 3.6.1 (2019-07-05)
#> os Ubuntu 18.04.2 LTS
#> system x86_64, linux-gnu
#> ui X11
#> language
#> collate en_US.UTF-8
#> ctype en_US.UTF-8
#> tz America/New_York
#> date 2019-09-13
#>
#> ─ Packages ──────────────────────────────────────────────────────────────
#> package * version date lib source
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.1)
#> backports 1.1.4 2019-04-10 [1] CRAN (R 3.6.1)
#> base64url 1.4 2018-05-14 [1] CRAN (R 3.6.1)
#> callr 3.3.1 2019-07-18 [1] CRAN (R 3.6.1)
#> cli 1.1.0 2019-03-19 [1] CRAN (R 3.6.1)
#> codetools 0.2-16 2018-12-24 [1] CRAN (R 3.6.1)
#> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.1)
#> data.table * 1.12.2 2019-04-07 [1] CRAN (R 3.6.1)
#> desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.1)
#> devtools 2.2.0 2019-09-07 [1] CRAN (R 3.6.1)
#> digest 0.6.20 2019-07-04 [1] CRAN (R 3.6.1)
#> drake * 7.6.1 2019-08-19 [1] CRAN (R 3.6.1)
#> DT 0.8 2019-08-07 [1] CRAN (R 3.6.1)
#> ellipsis 0.2.0.1 2019-07-02 [1] CRAN (R 3.6.1)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 3.6.1)
#> filelock 1.0.2 2018-10-05 [1] CRAN (R 3.6.1)
#> fs 1.3.1 2019-05-06 [1] CRAN (R 3.6.1)
#> future 1.14.0 2019-07-02 [1] CRAN (R 3.6.1)
#> globals 0.12.4 2018-10-11 [1] CRAN (R 3.6.1)
#> glue 1.3.1 2019-03-12 [1] CRAN (R 3.6.1)
#> highr 0.8 2019-03-20 [1] CRAN (R 3.6.1)
#> htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.6.1)
#> htmlwidgets 1.3 2018-09-30 [1] CRAN (R 3.6.1)
#> igraph 1.2.4.1 2019-04-22 [1] CRAN (R 3.6.1)
#> knitr 1.24 2019-08-08 [1] CRAN (R 3.6.1)
#> listenv 0.7.0 2018-01-21 [1] CRAN (R 3.6.1)
#> magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.1)
#> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.1)
#> pillar 1.4.2 2019-06-29 [1] CRAN (R 3.6.1)
#> pkgbuild 1.0.5 2019-08-26 [1] CRAN (R 3.6.1)
#> pkgconfig 2.0.2 2018-08-16 [1] CRAN (R 3.6.1)
#> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.6.1)
#> prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.6.1)
#> processx 3.4.1 2019-07-18 [1] CRAN (R 3.6.1)
#> ps 1.3.0 2018-12-21 [1] CRAN (R 3.6.1)
#> R6 2.4.0 2019-02-14 [1] CRAN (R 3.6.1)
#> Rcpp 1.0.2 2019-07-25 [1] CRAN (R 3.6.1)
#> remotes 2.1.0 2019-06-24 [1] CRAN (R 3.6.1)
#> rlang 0.4.0 2019-06-25 [1] CRAN (R 3.6.1)
#> rmarkdown 1.15 2019-08-21 [1] CRAN (R 3.6.1)
#> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.1)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.1)
#> storr 1.2.1 2018-10-18 [1] CRAN (R 3.6.1)
#> stringi 1.4.3 2019-03-12 [1] CRAN (R 3.6.1)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 3.6.1)
#> testthat 2.2.1 2019-07-25 [1] CRAN (R 3.6.1)
#> tibble 2.1.3 2019-06-06 [1] CRAN (R 3.6.1)
#> txtq 0.1.5 2019-08-19 [1] CRAN (R 3.6.1)
#> usethis 1.5.1 2019-07-04 [1] CRAN (R 3.6.1)
#> withr 2.1.2 2018-03-15 [1] CRAN (R 3.6.1)
#> xfun 0.9 2019-08-21 [1] CRAN (R 3.6.1)
#> yaml 2.2.0 2018-07-25 [1] CRAN (R 3.6.1)
#>
#> [1] /home/landau/R/R-3.6.1/library
Update
Please run this reprex yourself and post the result:
reprex::reprex({
library(drake)
# Write script files
package_code <- c(
"library(data.table)",
"library(drake)"
)
function_code <- c(
"make_data <- function(download_date){",
" data.table(",
" dt = download_date,",
" x = runif(n = 1e5),",
" y = sample(letters,size = 1e5,replace = T)",
" )",
"}"
)
plan_code <- c(
"plan <- drake_plan(",
" download_date = \"2019-09-08\",",
" x = make_data(download_date)",
")"
)
drake_code <- c(
"source(\"packages.R\")",
"source(\"functions.R\")",
"source(\"plan.R\")",
"drake_config(",
" plan,",
" verbose = 2,",
" jobs = 3L,",
" lock_envir = FALSE,",
" parallelism = \"future\",",
" console_log_file = \"r_make.log\"",
")"
)
writeLines(package_code, "packages.R")
writeLines(function_code, "functions.R")
writeLines(plan_code, "plan.R")
writeLines(drake_code, "_drake.R")
# Check that we wrote the scripts properly
cat(readLines("packages.R"), sep = "\n")
cat(readLines("functions.R"), sep = "\n")
cat(readLines("plan.R"), sep = "\n")
cat(readLines("_drake.R"), sep = "\n")
# Is there already .drake/ cache somewhere?
list.files()
find_cache()
# If so, this reprex is not clean.
for (i in seq_len(10)) {
clean(destroy = TRUE)
}
# Get ready
source("packages.R")
source("functions.R")
source("plan.R")
config <- drake_config(plan)
# Run make() and r_make()
make(
plan,
verbose = 2,
parallelism = "future",
jobs = 4L,
lock_envir = FALSE,
console_log_file = "make.log"
)
outdated(config)
r_make()
r_outdated()
make(
plan,
verbose = 2,
parallelism = "future",
jobs = 4L,
lock_envir = FALSE,
console_log_file = "make.log"
)
outdated(config)
r_make()
r_outdated()
# Check the log files to see what triggered the targets.
cat(readLines("make.log"), sep = "\n")
cat(readLines("r_make.log"), sep = "\n")
}, si = TRUE, venue = "so")
Upvotes: 1
Related Questions
- R's Makevars: PKG_CXXFLAGS vs. PKG_CXX11FLAGS
- Using the import package with drake
- r drake file paths on windows messed up
- Using rvest with drake: external pointer is not valid error
- Use a dedicated environment for drake, with r_make()
- "Missing files for target" error using Drake for Rmd
- Halting drake plan makes it rebuild targets it already had built previously
- Package build ignores Makevars flags
- building R from source: make error 1
- make always rebuilds Makefile targets