CODEWITHSUNDEEP
java
Enzo
Enzo

Reputation: 597

Kullback Leibler JAVA: how normalize distributions

I have this method in java to calculate the KL of probability distributions:

 public static double klDivergence(double[] p1, double[] p2) {
  double klDiv = 0.0;
  for (int i = 0; i < p1.length; ++i) {
    if (p1[i] == 0) { continue; }
    if (p2[i] == 0.0) { continue; } 

  klDiv += p1[i] * Math.log( p1[i] / p2[i] );
  }

  return klDiv; 
}


public static void main(String[] args) {

 double[] d1=new double[2];
 double[] d2=new double[2];

 d1[0]=0.23654;
 d1[1]=0.56565;  

 d2[0]=0.23654;
 d2[1]=0.89312;


 double symmetric= (klDivergence(d1, d2)+klDivergence(d2, d1))/2;
}

Should I normalize the probability distributions before calculating the symmetric KL? If you like?

Upvotes: 2

Views: 1218

Answers (1)

John
John

Reputation: 896

The Kullback-Leibler divergence is defined on discrete probability distributions. Those are per definition normalized to 1. Furthermore, without normalization d1[0]=0;d1[1]=1 and d2[0]=0;d2[1]=2 would result in a Kullback-Leibler divergence other than zero. So yes, you should normalize your distributions.

Upvotes: 1

Related Questions