CODEWITHSUNDEEP
mathmachine-learningartificial-intelligenceentropy
Sunny
Sunny

Reputation: 13

How to distinguish low entropy and high entropy with the information produced using shannon entropy

I calculated an entropy level of user's behaviour for its possible states of occurence (H:=Home, w:=Work or E:=Elsewhere) for a day. Say a user A has its possible states for each hour of a day as {H,H,H,H,H,H,H,H,H,W,W,W,W,W,W,W,W,W,E,E,H,H,H,H}

P(H)=13/24=0.54

P(W)=9/24=0.38

P(E)=2/24=0.08

Using shannon's entropy, I wanted to calculate the entropy level of a user to figure out how predictable the user is.

H(A)=(-P(H)*log2(H)) + (-P(W)*log2(W)) + (-P(E)*log2(E))

=0.48 + 0.53 + 0.29

=1.3

How to distinguish this information in bits as low entropy or high entropy?

Upvotes: 1

Views: 1015

Answers (1)

4pie0
4pie0

Reputation: 29734

A maximum entropy is achieved when all events are equally probable, and thus the outcome has highest uncertainty

enter image description here

In your case maximum entropy is:

Hn = log2(24) = 4.5849625007

The minimum is 0. It is up to you to decide if the result can be classified as high or low entropy.

Upvotes: 1

Related Questions