CODEWITHSUNDEEP
rsystem-callspandoc
Josh
Josh

Reputation: 1337

Windows system call fails while building the book "R Packages"

I'm trying to build Hadley's book "R Packages" from source. I have downloaded and unzipped the source and am have combined the code from Getting started (with some modifications) and the code in r-pkgs-first-edition-freeze\book\build-book.r to build the book (original code at the of this post). The first part of the code from Getting started works as expected:

# install.packages(c("devtools", "roxygen2", "testthat", "knitr"))      # orginal code
library(devtools); library(roxygen2); library(testthat); library(knitr) # what I'm using
# install.packages("rstudioapi")                                        # orginal code
library(rstudioapi)                                                     # what I'm using
rstudioapi::isAvailable("0.99.149")
  [1] TRUE                                                              # expected result

In the next part, has_devel() doesn't give the verbose output described in Getting started, but it also doesn't throw an error, and .Last.value == TRUE, as expected, so I think I'm OK:

devtools::install_github("r-lib/devtools")
has_devel()
.Last.value
  [1] TRUE

The last part of the code from Getting started also works. Many of my packages are newer versions compared to what is listed in Getting started, but I don't think this is the source of my subsequent problems.

library(roxygen2)
library(testthat)
devtools::session_info()
  ─ Session info ───────────────────────────────────────────────────────────
  setting  value                       
  version  R version 3.5.1 (2018-07-02)
  os       Windows 7 x64 SP 1          
  system   x86_64, mingw32             
  ui       RStudio                     
  language (EN)                        
  collate  English_United States.1252  
  tz       America/Chicago             
  date     2019-04-10                  

  ─ Packages ───────────────────────────────────────────────────────────────
  package       * version    date       source                             
  assertthat      0.2.0      2017-04-11 CRAN (R 3.5.1)                     
  backports       1.1.2      2017-12-13 CRAN (R 3.5.0)                     
  callr           2.0.4      2018-05-15 CRAN (R 3.5.1)                     
  cli             1.0.0      2017-11-05 CRAN (R 3.5.1)                     
  clisymbols      1.2.0      2017-05-21 CRAN (R 3.5.1)                     
  commonmark      1.5        2018-04-28 CRAN (R 3.5.1)                     
  crayon          1.3.4      2017-09-16 CRAN (R 3.5.1)                     
  desc            1.2.0      2018-05-01 CRAN (R 3.5.1)                     
  devtools      * 2.0.2.9000 2019-04-11 Github (r-lib/devtools@ab6c878)    
  digest          0.6.15     2018-01-28 CRAN (R 3.5.1)                     
  evaluate        0.11       2018-07-17 CRAN (R 3.5.1)                     
  fs              1.2.5      2018-07-30 CRAN (R 3.5.1)                     
  glue            1.3.0      2018-07-17 CRAN (R 3.5.1)                     
  htmltools       0.3.6      2017-04-28 CRAN (R 3.5.1)                     
  knitr         * 1.20       2018-02-20 CRAN (R 3.5.1)                     
  magrittr        1.5        2014-11-22 CRAN (R 3.5.1)                     
  memoise         1.1.0      2017-04-21 CRAN (R 3.5.1)                     
  oldbookdown   * 0.1        2019-04-11 Github (hadley/oldbookdown@0ffc6fb)
  pkgbuild        1.0.3      2019-04-11 Github (r-lib/pkgbuild@79cb7a0)    
  pkgload         1.0.1.9000 2019-04-11 Github (r-lib/pkgload@0ef4f58)     
  prettyunits     1.0.2      2015-07-13 CRAN (R 3.5.1)                     
  processx        3.1.0      2018-05-15 CRAN (R 3.5.1)                     
  R6              2.2.2      2017-06-17 CRAN (R 3.5.0)                     
  Rcpp            0.12.18    2018-07-23 CRAN (R 3.5.1)                     
  remotes         2.0.3      2019-04-09 url                                
  RevoUtils     * 11.0.1     2018-08-01 local                              
  RevoUtilsMath * 11.0.0     2018-08-01 local                              
  rlang           0.2.1      2018-05-30 CRAN (R 3.5.1)                     
  rmarkdown     * 1.10       2018-06-11 CRAN (R 3.5.1)                     
  roxygen2      * 6.1.0      2018-07-27 CRAN (R 3.5.1)                     
  rprojroot       1.3-2      2018-01-03 CRAN (R 3.5.1)                     
  rstudioapi    * 0.7        2017-09-07 CRAN (R 3.5.1)                     
  sessioninfo     1.0.0      2017-06-21 CRAN (R 3.5.1)                     
  stringi         1.2.4      2018-07-20 CRAN (R 3.5.1)                     
  stringr         1.3.1      2018-05-10 CRAN (R 3.5.1)                     
  testthat      * 2.0.0      2017-12-13 CRAN (R 3.5.1)                     
  usethis       * 1.4.0      2018-08-14 url                                
  withr           2.1.2      2018-03-15 CRAN (R 3.5.1)                     
  xml2            1.2.0      2018-01-24 CRAN (R 3.5.1)                     
  yaml            2.2.0      2018-07-25 CRAN (R 3.5.1)

The first part of the code in build-book.r works for me:

install_github("hadley/oldbookdown@0ffc6fb")  # code I needed to add
library(oldbookdown)
library(rmarkdown)
# Render chapters into tex  ------------------------------------------------
needs_update <- function(src, dest) {
  if (!file.exists(dest)) return(TRUE)
  mtime <- file.info(src, dest)$mtime
  mtime[2] < mtime[1]
}
render_chapter <- function(src) {
  dest <- file.path("book/tex/", gsub("\\.rmd", "\\.tex", src))
  if (!needs_update(src, dest)) return()
  message("Rendering ", src)
  command <- bquote(rmarkdown::render(.(src), oldbookdown::tex_chapter(),
    output_dir = "book/tex", quiet = TRUE, env = globalenv()))
  writeLines(deparse(command), "run.r")
  on.exit(unlink("run.r"))
  source_clean("run.r")
}
source_clean <- function(path) {
  r_path <- file.path(R.home("bin"), "R")
  cmd <- paste0(shQuote(r_path), " --quiet --file=", shQuote(path))
  out <- system(cmd, intern = TRUE)
  status <- attr(out, "status")
  if (is.null(status)) status <- 0
  if (!identical(as.character(status), "0")) {
    stop("Command failed (", status, ")", call. = FALSE)
  }
}
chapters <- dir(".", pattern = "\\.rmd$")

But the code fails when I try to actually render the chapters:

lapply(chapters, render_chapter)
  Rendering check.rmd
  Error: Command failed (1)
  In addition: Warning message:
    In system(cmd, intern = TRUE) :
    running command '"C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64/R" --quiet --file="run.r"' had status 1
  Called from: source_clean("run.r")
  Browse[1]

The system call fails, but since I don't know what the Windows call "C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64/R" --quiet --file="run.r" is supposed to do, I have no idea how to trouble shoot this. I tried changing r_path <- file.path(R.home("bin"), "R") to r_path <- file.path(R.home("bin")) (since there is no "R" directory in C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64), but I still get the error below:

lapply(chapters, render_chapter)
  Rendering check.rmd
  Error in system(cmd, intern = TRUE) : 
    '"C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64"' not found

I also tried changing file.path(R.home("bin"), "R") to file.path(R.home("bin"), "R"), since there is a "R.exe" executable file in this directory, also with no luck. Any guidance would be appreciated.

Additional information, in response to @onlyphantom: it appears that the broken path is found by R.home("bin") has also affected other system variables, including R_DOC_DIR and R_HOME. But PATH points to the correct location, C:\Program Files\Microsoft\R Open\R-3.5.1\bin\x64.

file.path(R.home("bin"))
  [1] "C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64"
Sys.getenv()    # returned values below are redacted
  APPDATA                              C:\Users\JT\AppData\Roaming
  HOME                                 C:/Users/JT/R/Projects
  HOMEDRIVE                            C:
    HOMEPATH                             \Users\JT\R
  LOCALAPPDATA                         C:\Users\JT\AppData\Local
  PATH                                 C:\Program Files\Microsoft\R Open\R-3.5.1\bin\x64;...
  ProgramData                          C:\ProgramData
  ProgramFiles                         C:\Program Files
  ProgramFiles(x86)                    C:\Program Files (x86)
  R_DOC_DIR                            C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/doc
  R_HOME                               C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1
  R_LIBS_USER                          C:/Program Files/Microsoft/R Open/R-3.5.1/library
  R_USER                               C:/Users/JT/R

I tried uninstalling and then reinstalling R Open, but still get this:

file.path(R.home("bin"), "R")
  [1] "C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64/R"

I tried switching to regular R (instead of R Open) and continued to have the same problem.

Windows Command Line Problem: I walked step-by-step through the R code to find the exact location of the problem. It occurs when render_chapter() calls source_clean("run.r"), which eventually calls out <- system(cmd, intern = TRUE). cmd (for the first object in chapters) is "C:\Program Files\R\R-3.5.3\bin\x64\R" --quiet --file="run.r". When I open a Windows command prompt and run this code I get the following:

C:\>cd C:\Users\JT\R\Books\r-pkgs-first-edition-freeze

C:\Users...freeze>"C:\Program Files\R\R-3.5.3\bin\x64\R" --quiet --file="run.r"
    > rmarkdown::render("check.rmd", oldbookdown::tex_chapter(), output_dir = "book/tex",
    +     quiet = TRUE, env = globalenv())
    --chapters has been removed. Use --top-level-division=chapter instead.
    Try pandoc.exe --help for more information.
    Error: pandoc document conversion failed with error 2
    Execution halted

When I run pandoc.exe --help I see an entry --top-level-division=section|chapter|part. The switch away from --chapters happened after pandocs v1.17.2, so I uninstalled the current version and installed pandocs v1.17.2. I think this fixed the problem, but created a new problem. I now get this error (with either the short or long form of the path name):

C:\Users...freeze>"C:/PROGRA~1/R/R-35~1.3/bin/x64/R" --quiet --file="run.r"
 #or#
C:\Users...freeze>"C:\Program Files\R\R-3.5.3\bin\x64\R" --quiet --file="run.r"
> rmarkdown::render("check.rmd", oldbookdown::tex_chapter(), output_dir = "book/
tex",
+     quiet = TRUE, env = globalenv())
pandoc.exe: Unknown reader: markdown_style
Error: pandoc document conversion failed with error 7
Execution halted

My assumption is that the old version of pandocs is not compatible with the new markdown_style. So I'm going to try installing the newest version of pandocs and then edit the run.r file to change --chapters to --top-level-division=section|chapter|part. If that's a stupid idea, please let me know.

Upvotes: 1

Views: 213

Answers (2)

Josh
Josh

Reputation: 1337

Turns out, there were two problems, both of which gave essentially the same error when lapply(chapters, render_chapter) was called. And it had nothing to do with the short version of the path (i.e. C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64/R).

First, oldbookdown::tex_chapter() returned a list that was not compatible with the latest version of pandocs. So I changed the render_chapter function as follows:

render_chapter <- function(src) {
  dest <- file.path("book/tex/", gsub("\\.rmd", "\\.tex", src))
  if (!needs_update(src, dest)) return()
  message("Rendering ", src)
  # "changed oldbookdown::tex_chapter()" to "temp"
  command <- bquote(rmarkdown::render(.(src), temp, output_dir = "book/tex", quiet = TRUE, env = globalenv()))
  # added definition for temp (with appropriate changes to 
  # oldbookdown::tex_chapter()) to the writeLines() call
  writeLines(
    c(
      "temp <- oldbookdown::tex_chapter()",
      'temp$pandoc$from <- "markdown"',
      'temp$pandoc$args[1] <- "--top-level-division=chapter"',
      deparse(command)
    ),
    "run.r"
  )
  on.exit(unlink("run.r"))
  source_clean("run.r")
}

Second, pandocs was calling multiple R packages in the background and failing if they were not installed. So I had to call render_chapter on each object in chapters individually to identify and install the missing packages.

Now system("xelatex -interaction=batchmode r-packages ") is failing, but that's a topic for a different question.

Upvotes: 0

onlyphantom
onlyphantom

Reputation: 9603

The problem lies with a wrong R home directory. C:/PROGRA~1 is just the short form of path elements, and really points to C:/PROGRAM FILES/....

The same for R-35~1.1, it points to /R/R-3.5.1 or something. It's just a "short name".

The system call fails, but since I don't know what the Windows call "C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64/R" --quiet --file="run.r" is supposed to do, I have no idea how to trouble shoot this.

Your script includes the following code:

source_clean <- function(path) {
  r_path <- file.path(R.home("bin"), "R")
  ...
}

The value of r_path at this point is C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64/R, which may have resulted from a previous (or failed) installation of R Open.

I tried changing r_path <- file.path(R.home("bin"), "R") to r_path <- file.path(R.home("bin")) (since there is no "R" directory in C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64), but I still get the error below: "C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64"' not found

What you can do instead, is to set r_path directly to the directory containing your R installation. Open up R (in Rstudio or R console) and verify this:

file.path(R.home("bin"))
[1] "/Library/Frameworks/R.framework/Resources/bin"

Sys.getenv("R_HOME")
"/Library/Frameworks/R.framework/Resources"

The R.home function returns the R home directory. If you navigate to that exact location on your machine, you should see that's where your R installation is that.

Setting R environment variable on Windows

Let's say you find your R installation here C:\Program Files\R\R-3.3.2\bin\x86, then do the following:

  1. Open Start Menu, type in "View advanced system settings" and hit "Environment variables".
  2. Under "System variables", select Path and click on edit
  3. Click “New”, and add the folder address for R to there (C:\Program Files\R\R-3.3.2\bin\x86)

Refer also to this answer and this for tips to set the path of your R home.

Upvotes: 2

Related Questions