CODEWITHSUNDEEP
rloopstime
Felipe Alvarenga
Felipe Alvarenga

Reputation: 2652

Pausing a loop for specific time at a specific time in R

I have to run a long loop that updates some data and stores it in my company's server. The problem is that the company runs a back-up routine at midnight, and, for that, they shutdown the server for around 15 minutes.

So, given that I have to write down a file for every iteration, when the server goes down it breaks the loop.

I managed to circumvent the problem by writing the loop as follows

for(i in bills.list){
  url = paste0("ulalah",i,"/")
  # Download the data
  bill.result <- try(getURL(url)) # if there is an error try again
  while(class(bill.result)=="try-error"){
    Sys.sleep(1)
    bill.result <- try(getURL(url))
  }

# if iteration is between 23:59:00 and 23:59:40 wait 17 min to restart the loop
  if(as.numeric(format(Sys.time(), "%H%M%S")) > 235900 &
     as.numeric(format(Sys.time(), "%H%M%S")) < 235940){

    Sys.sleep(1020)

  }
  # Write the page to local hard drive
  write(bill.result, paste0("bill", i, ".txt"))

  # Print progress of download
  cat(i, "\n")
}

Problem is, by evaluating the time at all iterations, I lose some precious time. Any more efficient thoughts?

Upvotes: 3

Views: 325

Answers (1)

Oo.oO
Oo.oO

Reputation: 13375

I think you could simply try to store date. If it fails, it might be you are inside backup window

store <- function(data, retry = 10) {
  while(retry > 0) {
    result <- try(write(data, "/some_broken_place"))
    if(class(result) == "try-error") {
      # it might be we are in the backup window 
      cat("I will sleep if it turns out that it's backup time")
      if(as.numeric(format(Sys.time(), "%H%M%S")) > 235900 &
         as.numeric(format(Sys.time(), "%H%M%S")) < 235940){
        Sys.sleep(1020)
      }
      retry <- retry - 1
    }
  }
  if(retry == 0) {
    cat("Very, very bad situation - no chance to store data")
  }
}

Upvotes: 1

Related Questions