Reputation: 1299
Is there a new solution for downloading package and dependencies for a given R version
I have seen the question asked here from 2018. I'm wondering if there is a better answer today.
Our work computers are bedeviled by an entire IT security department that seems to exist to make them useless. We are allowed to run R 3.6.3 (4.x hasn't been approved yet). We cannot connect to CRAN from behind the corporate firewall. In the past that meant we took our laptops home to install packages. But now we have a download monitor that blocks CRAN downloads even when we're on our own wi-fi.
I was attempting to get around this by downloading the package .zip files on a personal machine, transferring them via CD, and then installing with repos=NULL. I used this code
getPackages <- function(packs){
packages <- unlist(
tools::package_dependencies(packs, available.packages(),
which=c("Depends", "Imports"), recursive=TRUE)
)
packages <- union(packs, packages)
packages
}
to get the dependencies for the package I wanted and downloaded them all with download.packages(). I was trying to install tidyquant so it turned out there were 113 dependencies.
But of course this downloads the latest version of all 113 packages, several of which it turned out were incompatible with R 3.6.3.
The solution I referenced above suggested finding a compatible version and then using
install_version("<package name>",
version = "<version number>",
repos = "http://cran.us.r-project.org")
But that would involve manually searching all 113 dependencies...plus I'm looking for a general purpose solution that I can share with the ~10 other people in my office that all have this issue now.
I'm hoping that since 2018 a better solution may have presented itself?
Update: Based on answers below I tried
tmp <- tempfile()
dir.create(tmp)
checkpoint::checkpoint(snapshot_date = "2020-04-01",
r_version = "3.6.3",
checkpoint_location = tmp,
scan_now = FALSE)
packages <- getPackages(c("tidyquant"))
download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
I get
Running create_checkpoint in the home directory may result
in checkpointing very many packages. Continue? (Y/n) Y
Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
Warning messages:
1: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
Specified R version not the same as current R version
2: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
Specified R version not the same as current R version
>
> packages <- getPackages(c("tidyquant"))
>
> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
no package ‘tidyquant’ at the repositories
Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
no package ‘lubridate’ at the repositories
Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
no package ‘PerformanceAnalytics’ at the repositories
And the "no package" warnings go on for all 113 dependencies.
I think the key here is
Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
Obviously there wouldn't be a 4.1 repository in April 2020. I assume it's trying that since the machine is running 4.1. So it seems we're tantalizingly close....
If it helps
> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252
system code page: 65001
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.1.2 tools_4.1.2 checkpoint_1.0.2
Upvotes: 3
Views: 817
Answers (1)
Reputation: 11306
I'm not sure if it completely addresses your needs, but package checkpoint seems appropriate here. It allows you to download source packages from a snapshot of CRAN taken at a specified date, going back to 2014-09-17. R 4.0.0 was released around 2020-04-24, so the snapshot from 2020-04-01 should work for your purposes.
Here is a reproducible example:
tmp1 <- tempfile()
dir.create(tmp1)
cwd <- setwd(tmp1)
getOption("repos")
## CRAN
## "https://cloud.r-project.org/"
available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
## Version Repository
## lattice "0.20-45" "https://cloud.r-project.org/src/contrib"
## Matrix "1.4-0" "https://cloud.r-project.org/src/contrib"
## nlme "3.1-155" "https://cloud.r-project.org/src/contrib"
download.packages("Matrix", ".", type = "source")
## trying URL 'https://cloud.r-project.org/src/contrib/Matrix_1.4-0.tar.gz'
## Content type 'application/x-gzip' length 2849865 bytes (2.7 MB)
## ==================================================
## downloaded 2.7 MB
##
## [,1] [,2]
## [1,] "Matrix" "./Matrix_1.4-0.tar.gz"
tmp2 <- tempfile()
dir.create(tmp2)
checkpoint::checkpoint(snapshot_date = "2020-04-01",
r_version = "3.6.3",
checkpoint_location = tmp2,
scan_now = FALSE)
## Creating checkpoint directory /var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T//RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
## Using checkpoint directory /private/var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T/RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
## Warning messages:
## 1: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
## Specified R version not the same as current R version
## 2: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
## Specified R version not the same as current R version
getOption("repos")
## CRAN
## "https://mran.microsoft.com/snapshot/2020-04-01"
available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
## Version Repository
## lattice "0.20-40" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
## Matrix "1.2-18" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
## nlme "3.1-145" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
download.packages("Matrix", ".", type = "source")
## trying URL 'https://mran.microsoft.com/snapshot/2020-04-01/src/contrib/Matrix_1.2-18.tar.gz'
## Content type 'application/octet-stream' length 1871705 bytes (1.8 MB)
## ==================================================
## downloaded 1.8 MB
##
## [,1] [,2]
## [1,] "Matrix" "./Matrix_1.2-18.tar.gz"
setwd(cwd)
unlink(c(tmp1, tmp2), recursive = TRUE)
The warnings about version mismatch occur if you are not actually running R 3.6.3. They can be ignored if you are only downloading source packages, with the intention of installing them on another machine actually running 3.6.3.
You can take a look at the package README and ?checkpoint for more details.
Update
If you are trying to download binary packages (.zip for Windows, .tgz for macOS) rather than source packages (.tar.gz), then checkpoint can get you into trouble. By default, download.packages and friends use contrib.url(repos, type) to construct a URL to search for package binaries.
contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "win.binary")
## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1"
contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "mac.binary")
## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1"
But there is nothing at either URL. That is (in part) because contrib.url appends the R version that you are currently running, which might not have existed on your snapshot date. Hence:
download.packages("Matrix", ".", type = "win.binary")
## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
## cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
## Warning in download.packages("Matrix", ".", type = "win.binary") :
## no package 'Matrix' at the repositories
## [,1] [,2]
download.packages("Matrix", ".", type = "mac.binary")
## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1:
## cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1/PACKAGES'
## Warning in download.packages("Matrix", ".", type = "mac.binary") :
## no package 'Matrix' at the repositories
## [,1] [,2]
The URLs that you actually need are:
## Windows
"https://cran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/3.6"
## macOS, OS X, whatever
"https://cran.microsoft.com/snapshot/2020-04-01/bin/macosx/el-capitan/contrib/3.6"
The best way forward, in this case, may be to avoid checkpoint altogether and directly pass the valid URL to available.packages and download.packages:
pkg <- "tidyquant"
contriburl <- "https://cran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/3.6"
db <- available.packages(contriburl)
deps <- tools::package_dependencies(pkg, db, recursive = TRUE)[[pkg]]
download.packages(c(pkg, deps), ".", contriburl = contriburl, type = "win.binary")
Upvotes: 3
Related Questions
- How to update all packages for a new R version quickly and easily?
- Install a local R package with dependencies from CRAN mirror
- R package dependencies
- Request the latest CRAN package version by source when binaries not yet up
- Update a specific R package and its dependencies
- How to download the entire CRAN repository?
- How do different versions of R manage to seemingly automatically download different appropriate package versions from CRAN?
- Efficiently getting older versions of R packages
- Possible to state explicit versions of package dependencies?
- Is there an easy way of replicating a list of installed CRAN packages (with the exact versions)?