CODEWITHSUNDEEP
rmathematical-optimization
hxd1011
hxd1011

Reputation: 915

Is there any way to extract parameters and objective function for each iteration in R optimx

Suppose I have an optimization problem to solve using R, optimx. Is there any way I can extract the parameters and objective function values over time? 

f<-function(x){
  return(sum(abs(x)))
}

gr<-function(x){
  return(sign(x))
}
opt=optimx::optimx(runif(2),f,gr,method="BFGS")

The goal is trying to make such plot:

enter image description here

I think we can manually do it with Gradient Decent with following code, but how I can I do it in optimx?

x=c(0.5,1.5)
alpha=0.1
max_iter=20
x_trace=matrix(rep(0,max_iter*2),ncol=2)

for (i in 1:max_iter){
  x=x-alpha*gr(x)
  x_trace[i,]=x
}
f_trace=apply(x_trace,1,f)

Upvotes: 4

Views: 500

Answers (2)

cuttlefish44
cuttlefish44

Reputation: 6776

As far as I know, method="L-BFGS-B" can make a progress report of params. But the result doesn't have the report, so I kept the message and extracted the value. 

library(optimx); library(dplyr)

cap <- capture.output(optimx(runif(2), f, gr, method="L-BFGS-B", 
                             control=list(trace=6, REPORT=1)))
temp <- cap[grep("X =|X0 =", cap)]
d <- gsub("X0 = |X = |Cauchy X =  ", "", temp) %>% strsplit(" ") %>% 
  unlist() %>% as.numeric() %>% matrix(ncol=2, byrow=T)

plot(-2:2,-2:2, type="n", ann=F)
for(i in c(1,2,4)) polygon(c(-0.5,0,0.5,0, -0.5)*i, c(0, 0.5, 0, -0.5, 0)*i)
points(d, pch=letters[1:nrow(d)])

[edited]
As help says, source code (opt/lbfgs_bcm.shar) helps to exactly understand these values (@Roland commented, thanks !!). And using this approach with method="L-BFGS-B", you can get additional information about what values control=list(trace=6, REPORT=1) reports.

enter image description here

Upvotes: 2

Roland
Roland

Reputation: 132651

Create a side effect:

f<-function(x){
  .GlobalEnv$i <- get("i", envir = .GlobalEnv) + 1
  .GlobalEnv$log[get("i", envir = .GlobalEnv),] <- x
  return(sum(abs(x)))
}

gr<-function(x){
  return(sign(x))
}

library(optimx)
i <- 0
log <- matrix(numeric(100 * 2), ncol = 2)

opt <- optimx(c(0.8, -0.9),f,gr,method="BFGS")
log <- log[seq_len(i), ]

plot(log, type = "l", xlim = c(-2, 2), ylim = c(-1.2, 1.2))

resulting plot

Note that this includes all function calls, even those where the algorithm rejects the result and retries. control = list(trace = TRUE, REPORT = 1) lets optimx print the function values for accepted tries and you could capture.output this and use it to get only the parameters of these from log.

It would be better to change optimx to return all accepted attempts, but I'm not going to invest that kind of effort. You could ask Prof. Nash if he would be willing to do this, but if you don't have a compelling common use case, he probably is not going to either.

Upvotes: 5

Related Questions