CODEWITHSUNDEEP
matlabstatisticsprobabilityinformation-theory
Composer
Composer

Reputation: 255

Shannon's Entropy calculation

I have a probability distribution that defines the probability of occurrence of n possible states.

I would like to calculate the value of Shannon's entropy, in bits, of the given probability distribution.

Can I use wentropy(x,'shannon') to get the value and if so where can I define the number of possible states a system has?

Upvotes: 4

Views: 16346

Answers (1)

chappjc
chappjc

Reputation: 30579

Since you already have the probability distribution, call it p, you can do the following formula for Shannon Entropy instead of using wentropy:

H = sum(-(p(p>0).*(log2(p(p>0)))));

This gives the entropy H in bits.

p must sum to 1.

Upvotes: 8

Related Questions