CODEWITHSUNDEEP
rfunctionoptimization
stats_noob
stats_noob

Reputation: 5907

Automatically "recognizing" (calculus) derivatives

I am using the R programming language. Using the "optim" library and the "BFGS" optimization algorithm, I am interested in optimizing the following function (also called the "Rosenbrock Function"):

enter image description here

If you define this function and the derivative of this function, it is pretty straightforward to optimize with the "optim" library and the BFGS algorithm (note: the BFGS algorithm requires knowledge of the function's derivative):

fr <- function(x) {   ## Rosenbrock Banana function
    x1 <- x[1]
    x2 <- x[2]
    100 * (x2 - x1 * x1)^2 + (1 - x1)^2
}
grr <- function(x) { ## Gradient of 'fr'
    x1 <- x[1]
    x2 <- x[2]
    c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1),
       200 *      (x2 - x1 * x1))
}

res <- optim(c(-1.2,1), fr, grr, method = "BFGS")

> res
$par
[1] 1 1

$value
[1] 9.594956e-18

$counts
function gradient 
     110       43 

$convergence
[1] 0

$message
NULL

Suppose you are working with a high dimensional complicated function - the derivative of this function will be difficult to manually evaluate and then write a function for this derivative (i.e. an additional place where you can make a mistake). Are there any "automatic" ways in R, such that if you write a mathematical function - R can automatically "infer" the derivative of this function?

For instance, in a new R session - would there have been some way to run the BFGS algorithm without explicitly defining the derivative?

fr <- function(x) {   ## Rosenbrock Banana function
    x1 <- x[1]
    x2 <- x[2]
    100 * (x2 - x1 * x1)^2 + (1 - x1)^2
}


#pseudo code
res <- optim(c(-1.2,1), fr, ??? , method = "BFGS")

Does anyone know if something like this is possible? Can R automatically infer the derivative?

I thought of an approach where you could use a preexisting "numerical differentiation" function in R to approximate the derivative at each iteration, and then feed this approximation into the BFGS algorithm, but that sounds very complicated and unnecessary.

It would have been nice if R could somehow automatically infer the derivative of a function.

References:

Upvotes: 1

Views: 81

Answers (1)

Miff
Miff

Reputation: 7941

This is already built into the optim() function. If you don't specify the derivative it will be calculated numerically e.g.

fr <- function(x) {   ## Rosenbrock Banana function
  x1 <- x[1]
  x2 <- x[2]
  100 * (x2 - x1 * x1)^2 + (1 - x1)^2
}


optim(c(-1.2,1), fr, method = "BFGS")

# $par
# [1] 0.9998044 0.9996084
# 
# $value
# [1] 3.827383e-08
# 
# $counts
# function gradient 
# 118       38 
# 
# $convergence
# [1] 0
# 
# $message
# NULL

Note that (at least for this case) the solution is very close to that found using the analytical derivatives.

Upvotes: 1

Related Questions