CODEWITHSUNDEEP
pythonnumpy
Stefan Falk
Stefan Falk

Reputation: 25507

Why do I not get the same results for entropy()?

The documentation for scipy.stats.entropy says:

If qk is not None, then compute the Kullback-Leibler divergence S = sum(pk * log(pk / qk), axis=0).

print(np.sum(p * np.log(p / q), axis=0))
print(entropy(p, q))

However, I am getting two different values here:

0.510046080249
0.353272019382

In both cases the logarithm with base e should be used here. So why do I get two different results?

Upvotes: 0

Views: 281

Answers (1)

Max Shepherd
Max Shepherd

Reputation: 61

I think you're not normalising your p and q to 1. See below:

import numpy as np
from scipy.stats import entropy

p = np.asarray([1,2])
q = np.asarray([2,3])

p_norm = p/np.sum(p)
q_norm = q/np.sum(q)

print(np.sum(p_norm * np.log(p_norm / q_norm), axis=0))
print(entropy(p, q))

Upvotes: 1

Related Questions