R check for a package: optimize the way DLLs are built and checked

Question

I want to optimize my process for building a package. I have in pckgname/src some fortran code (f90):

pckgname/src/FortranFile1.f90   
pckgname/src/FortranFile2.f90   

I am under RStudio. When I build the package, it creates the src-i386 and src-x64 folders, inside which executable files in .o are produced

pckgname/src-i386/FortranFile1.o  
pckgname/src-i386/FortranFile2.o   
pckgname/src-x64/FortranFile1.o  
pckgname/src-x64/FortranFile2.o   

then dll files are produced into each of these folders from the .o files:

pckgname/src-i386/dllname.dll
pckgname/src-x64/dllname.dll

thereafter if I want to check the code successfully, I need to manually copy paste the dll into these two folders (in the previous version of the question i wrote code instead of dll which might have led to misunderstandings)

pckgname/inst/libs/x64/dllname.dll
pckgname/libs/X64/dllname.dll  

My question is: is it normal that I have to do this or is there a shorter way without having to copy paste by hand dllname.dll
into these two folders? It could be indeed a source of error.

NB: If i don't copy the dlls into the said folders I get the following error messages (translated from the French):

Error in inDL(x, as.logical(local), as.logical(now), ...) : 
impossible to load shared object 'C:/Users/username/Documents/pckgname/inst/libs/x64/dllname.dll':
LoadLibrary failure:  The specified module can't be found

Error in inDL(x, as.logical(local), as.logical(now), ...) : 
impossible to load shared object 'C:/Users/username/Documents/pckgname/libs/x64/dllname.dll':
LoadLibrary failure:  The specified module can't be found.
[...]
`cleanup` is deprecated 

Answer 1

No, it is not normal and there is a solution to this problem. Make use of Makevars.win. The reason for your problem is that .dlls are looking for dependencies in places defined by environment variable PATH and relative paths defined during the linking. Linking is being done when running the command R CMD INSTALL as it is stated in Mingw preferences plus some custom parameters defined in the file Makevars.win (Windows platform dependent). As soon as the resulting library is copied, the binding to the places where dependent .dlls were situated may become broken, so if you put dlls in a place where typically dependent libraries reside, such as, for instance, $(R_HOME)/bin/$(ARCH)/,

cp -f <your library relative path>.dll $(R_HOME)/bin/$(ARCH)/<your library>.dll

during the check R will be looking for your dependencies specifically there too, so you will not miss the dependencies. Very crude solution, but it worked in my case.

Answer 2

The short answer

Is it normal that I have to do this?

No. If path/to/package is the directory you are developing your package in, and you have everything set up for your package to call your Fortran subroutines correctly (see "The long answer"), you can run

R CMD build path/to/package

at the command prompt, and a tarball will be constructed for you with everything in the right place (note you will need Rtools for this). Then you should be able to run

R CMD check packagename_versionnumber.tar.gz

from the command prompt to check your package, without any problems (stemming from the .dll files being in the wrong place -- you may have other problems, in which case I would suggest asking a new question with the ERROR, WARNING, or NOTE listed in the question).

If you prefer to work just from R, you can even

devtools::check("path/to/package")

without having to run devtools::build() or R CMD build ("devtools::check()... [b]undles the package before checking it" -- Hadley's chapter on checking; see also Karl Broman's chapter on checking).

The long answer

I think your question has to do with three issues potentially:

1. The difference between directory structure of packages before and after they're installed. (You may want to read the "What is a package?" section of Hadley's Package structure chapter -- luckily R CMD build at the command prompt or devtools::build() in R will take care of that for you)
2. Using INSTALL vs. BUILD (from the comments to the original version of this answer)
3. The proper way to set up a package to call Fortran subroutines.

You may need quite a bit of advice on the process of developing R packages itself. Some good guides include (in increasing order of detail):

• Karl Broman's R package primer
• Hadley Wickham's R packages
• The Writing R Extensions manual

In particular, there are some details about having compiled code in an R package that you may want to be aware of. You may want to first read Hadley's chapter on compiled code (Broman doesn't have one), but then you honestly need to read most of the Writing R Extensions manual, in particular sections 1.1, 1.2, 1.5.4, and 1.6, and all of chapters 5 and 6.

In the mean time, I've setup a GitHub repository here that demonstrates a toy example R package FortranExample that shows how to correctly setup a package with Fortran code. The steps I took were:

1. Create the basic package structure using devtools::create("FortranExample").
2. Eliminate the "Depends" line in the DESCRIPTION, as it set a dependence on R >= 3.5.1, which will throw a warning in check (I have now also revised the "License" field to eliminate a warning about not specifying a proper license).
3. Make a src/ directory and add toy Fortran code there (it just doubles a double value).
4. Use tools::package_native_routine_registration_skeleton("FortranExample") to generate the symbol registration code that I placed in src/init.c (See Writing R Extensions, section 5.4).
5. Create a nice R wrapper for using .Fortran() to call the Fortran code (placed in R/example_function.R).
6. In that same file use the #' @useDynLib FortranExample Roxygen tag to add useDynLib(FortranExample) to the NAMESPACE file; if you don't use Roxygen, you can put it there manually (See Writing R Extensions 1.5.4 and 5.2).

Now we have a package that's properly set up to deal with the Fortran code. I have tested on a Windows machine (running Windows 8.1 and R 3.5.1) both the paths of running

R CMD build FortranExample
R CMD check FortranExample_0.0.0.9000.tar.gz

from the command prompt, and of running

devtools::check("FortranExample")

from R. There were no errors, and the only warning was the "License" issue mentioned above.

After cleaning up the after-effects of running devtools::check("FortranExample") (for some reason the cleanup option is now deprecated; see below for an R function to handle this for you inspired by devtools::clean_dll()), I used

devtools::install("FortranExample")

to successfully install the package and tested its function, getting:

FortranExample::example_function(2.0)
# [1] 4

The cleanup function I mentioned is

clean_source_dirs <- function(path) {
    paths <- file.path(path, paste0("src", c("", "-i386", "-x64")))
    file_pattern <- "\\.o|so|dll|a|sl|dyl"
    unlink(list.files(path = paths, pattern = file_pattern, full.names = TRUE))
}