CODEWITHSUNDEEP
precision
inmybrain
inmybrain

Reputation: 386

How to overcome underflow when obtaining a vector in simplex?

I'm normalizing a vector to make its sum 1 so that the vector will be used for a parameter in multinomial distribution.

Here's a toy example. Let x be a vector with positive values and sigma is very small positive number. When I run this R code, y is all zero due to underflow, so is its sum.

x <- rep(1, 10)
sigma <- 1e-3
y <- exp(-x/sigma) # 0's
prob <- y / sum(y) # NaNs

Do you have any idea to handle this?

Thanks in advance.

Upvotes: 0

Views: 251

Answers (1)

Ben Goodrich
Ben Goodrich

Reputation: 4980

Define the log_sum_exp function as 

log_sum_exp <- function(x) {
  m <- max(x)
  return(m + log(sum(exp(x - m))))
}

and then prob <- exp(y - log_sum_exp(y)) is the "softmax" of y.

Upvotes: 2

Related Questions