CODEWITHSUNDEEP
pythontensorflow
alighorbani
alighorbani

Reputation: 49

Multiplication of two small numbers with tensorflow

In some point of my project i have to multiply two small float like 8.696503446228892e-159 and 1.2425389522444519e-158 as i test in following code:

def a2(a,b):
  a = tf.cast(a, tf.float64)
  b = tf.cast(b, tf.float64)
  d = a*b
  return d

it will return 0 which cause lots of problem (because it is used in my loss function) any solution how can i multiply them?

Upvotes: 0

Views: 114

Answers (1)

Prune
Prune

Reputation: 77837

Handling large discrepancies in computational magnitude is a field of study in itself. The first-order way to do this is to write your evaluation code to detect the situation and re-order the operations so as to preserve significant bits of each result. For instance, let's simplify your names a bit:

tf.log(tf.linalg.det(temp_sigma) /
   (tf.sqrt(tf.linalg.det(sigma1) * tf.linalg.det(sigma2))))

turns into

log(det(A) / (sqrt(det(B) * det(c))))

The case you have is that det(B) and det(C) are barely above zero, but relatively near each other: the result of sqrt(det(B) * det(C)) will be close to either determinant.

Change the order of operations. For this instance, distribute the square root and do the divisions individually:

log(
    ( det(A) / sqrt(det(B)) ) / sqrt(det(C)) )

Does that get you moving along?

Upvotes: 1

Related Questions