CODEWITHSUNDEEP
rmemorymemory-management
asachet
asachet

Reputation: 6921

Is there a 2GB memory usage limit when R boots?

I have the following code for loading some data in my .Rprofile (which is a R script in my project folder running automatically when I switch to the project with Rstudio).

data_files <- list.files(pattern="\\.(RData|rda)$")
if("data.rda" %in% data_files) {
  attach(what="data.rda", 
         pos = 2)
  cat("The file 'data.rda' was attached to the search path under 'file:data.rda'.\n\n")
}

The data being loaded is relatively big:

                             Type       Size     PrettySize    Rows Columns
individual_viewings_26 data.frame 1547911120   [1] "1.4 Gb" 3685312      63
viewing_statements_all data.table  892316088   [1] "851 Mb" 3431935      38
weights                data.frame  373135464 [1] "355.8 Mb" 3331538      14
pet                    data.table   63926168    [1] "61 Mb"  227384      34

But I have 16 GB and I can allocate them:

> memory.limit()
[1] 16289

When my data was not as big, I did not have any issue. I recently saved some more data frames in data.rda and my R session suddenly fails at start-up (when I switch to the project in Rstudio and .Rprofile is executed):

Error: cannot allocate vector of size 26.2 Mb
In addition: Warning messages:
1: Reached total allocation of 2047Mb: see help(memory.size) 
2: Reached total allocation of 2047Mb: see help(memory.size) 
3: Reached total allocation of 2047Mb: see help(memory.size) 
4: Reached total allocation of 2047Mb: see help(memory.size)

I suspect that for some reason, the memory limit is set at 2GB at boot? Any way I can change that?

Edit: Added OS and software version

> sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Edit2: Just to clarify, I am able to load the data myself by running the code, I have plenty of available memory and the R process commonly uses up to 10GB during my daily work. The problem is, there is a apparently a 2GB memory limit when R boots and executes the .Rprofile...

Upvotes: 2

Views: 470

Answers (1)

asachet
asachet

Reputation: 6921

Yes, there is a 2GB limit when R starts, at least when the user profile (.Rprofile files and .First() functions) are executed.

Proof:

Content of Rprofile:

message("Available memory when .Rprofile is sourced: ", memory.limit())

.First <- function() {
  message("Available memory when .First() is called: ", memory.limit())
}

Output at startup

Available memory when .Rprofile is sourced: 2047
Available memory when .First() is called: 2047

Output of memory.limit once R has started

> memory.limit()
[1] 16289

Upvotes: 2

Related Questions