CODEWITHSUNDEEP
tensorflowtensorflow-probability
Sufeng Niu
Sufeng Niu

Reputation: 639

tensorflow distribution create probability greater than 1

I am using tensorflow distribution API for sampling, following is the sample code I am using, but I found the probability is greater than 1, then log probability is smaller than 0. I have tried both CPU and GPU, both produce this weird result. the tensorflow is 1.3.

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
from sklearn.datasets import load_boston
from sklearn.preprocessing import scale
from matplotlib import pyplot as plt

import numpy as np

learning_rate = 0.01

total_features, total_prices = load_boston(True)

# Keep 300 samples for training
train_features = scale(total_features[:300])
train_prices = total_prices[:300]

x = tf.placeholder(tf.float32, [None, 13])

l1 = tf.layers.dense(inputs=x, units=20, activation=tf.nn.elu)

l2 = tf.layers.dense(inputs=l1, units=20, activation=tf.nn.elu)

mu = tf.squeeze(tf.layers.dense(inputs=l2, units=1))
sigma = tf.squeeze(tf.layers.dense(inputs=l2, units=1))
sigma = tf.nn.softplus(sigma) + 1e-5

normal_dist = tf.contrib.distributions.Normal(mu, sigma)
samples = tf.squeeze(normal_dist._sample_n(1))

log_prob = -normal_dist.log_prob(samples)
prob = normal_dist.prob(samples)

sess = tf.Session()
sess.run(tf.global_variables_initializer())

avg_cost = 0.0

feed_dict = {x: train_features}

p = sess.run(prob, feed_dict)
lp = sess.run(log_prob, feed_dict)

The p is my probability output and lp is log probability

Thank you!

Upvotes: 2

Views: 965

Answers (1)

srvasude
srvasude

Reputation: 161

The functions .prob and .log_prob are the PDF and Log PDF of the normal distribution: https://en.wikipedia.org/wiki/Probability_density_function. Note that the PDF doesn't have to evaluate to a value between 0 and 1; It's integral over a range (which is related to the CDF) has to be between 0 and 1.

Consider the case where mu = 0 and sigma = 1e-4. If we use the PDF of the normal distribution: https://en.wikipedia.org/wiki/Normal_distribution, then PDF(0) ~= 4000! However, if we were to integrate the PDF and get the CDF (or use the CDF directly), then we will always get a value between 0 and 1.

Upvotes: 4

Related Questions