CODEWITHSUNDEEP
mathprecisiongaussianexponentialnumerical-stability
dontloo
dontloo

Reputation: 10865

Numerical problems of computing exponential numbers (from Gaussian PDFs)

I have a list of very small exponential values (for example exp(n), with n<-300), which are generated from Gaussian PDFs.

I want to compute how much each of them is proportional to the sum, for example like the following python-like pseudo-code does:

s = 0  # sum of all values
for n in exponents:
    s += exp(n)

for n in exponents:
    k = exp(n)/s  # I want to compute k for each n

The problem is, since the values of n are all very small, the summation s turns out to be zero sometimes, and I'll get the division-by-zero error.

I know one thing I can do is to add a constant value (say 300) to all n to prevent underflow, but it'll cause overflow in other cases.

How can I solve this?

I don't know whether I've expressed myself clearly, if any of these doesn't make sense or any grammar mistakes, please correct me. Thanks in advance.

Upvotes: 1

Views: 674

Answers (1)

WhatsUp
WhatsUp

Reputation: 1626

As you already observed, you can do it by subtracting a constant value m from all n.

To avoid overflow, do not choose a fixed m, such as m = - 300. Instead, choose m to be the maximum of all n. Then every normalized exponential value will be at most 1, hence the normalized sum will be at most the number of exponents, which should be reasonably small.

Upvotes: 1

Related Questions