CODEWITHSUNDEEP
r
Michael
Michael

Reputation: 1839

Closing read.table connections in R

I am running a script which load multiple files. Upon reaching 50 loaded files I get an error "all connections are in use".

I figured I have to close the connections but I encounter the following problem.

con = file(paste('/home/rstudio/userstats/',cuserid,'.tsv',sep=""))
userstats_current = read.table(con, sep="\t",  header=0, quote="", stringsAsFactors=F)
close(con)
Error in close.connection(con) : invalid connection

If I type the following however everything works fine:

con = file(paste('/home/rstudio/userstats/',cuserid,'.tsv',sep=""))
close(con)

Does something happen to connections when read.table is applied and how can I manage to close these connections?

UPDATE

Thank you for your responses. The issue is that when I run a foreach loop even with one core after a while I keep getting this all connections error.

registerDoMC(2)
matrix <- foreach(i=1:nrow(sample), .combine=rbind) %dopar% {....}

Upvotes: 1

Views: 3919

Answers (2)

Robert Casey
Robert Casey

Reputation: 1581

You can run into a similar kind of error when making http calls via the url() command

read.table(url("http://...."),....)

This can happen when you attempt to connect but get a 500 Server Error. In this case, read.table may not properly close the connection. After many loops of this kind, you will accumulate http CLOSE_WAIT sockets, which you can view with 'netstat -a', resulting in the 'all connections in use' error.

The solution for this is to make use of the RCurl package to carry out your URL connections, which is described in this Stack Overflow issue:

Stack Overflow: read data from internet

Upvotes: 0

Michael
Michael

Reputation: 1839

The error was caused by running read.table when the file wouldn't exist. Upon multiple read.table requests for files that don't exist on the disc connections are not released (unlike when files do exist).

To overcome the problem I used if (file.exists(filename)){read.table(filename)} and that seems to have fixed the issue. Thank you all for helping me resolve this.

Upvotes: 2

Related Questions