CODEWITHSUNDEEP
r
tan
tan

Reputation: 1569

Jensen Shannon divergence in R

I am new to R and was trying to find a function which calculates JS divergence in R. I can see that R has KLdiv for calculating KL divergence, but is there anything available for JS divergence?

Upvotes: 6

Views: 11160

Answers (3)

rapture
rapture

Reputation: 403

I was looking for a simple implementation of the JS divergence rather than an R library. Since I did not see one in any of the responses, I came up with the one below.

Assuming we have the following input distributions:

# p & q are distributions so their elements should sum up to 1
p <- c(0.00029421, 0.42837957, 0.1371827, 0.00029419, 0.00029419,
       0.40526004, 0.02741252, 0.00029422, 0.00029417, 0.00029418)

q <- c(0.00476199, 0.004762, 0.004762, 0.00476202, 0.95714168,
       0.00476213, 0.00476212, 0.00476202, 0.00476202, 0.00476202)

The Jensen-Shannon divergence would be:

n <- 0.5 * (p + q)
JS <- 0.5 * (sum(p * log(p / n)) + sum(q * log(q / n)))
> JS
[1] 0.6457538

For more than 2 distributions (which has already been discussed here) we need a function to compute the Entropy:

H <- function(v) {
      v <- v[v > 0]
      return(sum(-v * log(v)))
}

Then the JS divergence would be:

JSD <- function(w, m) {
  return(H(m %*% w) - apply(m, 2, H) %*% w)
}

> JSD(w = c(1/3, 1/3, 1/3), m = cbind(p, q, n))
          [,1]
[1,] 0.4305025

Where w is a vector of weights which should sum up to 1 and m is a matrix with the input distributions as columns.

Upvotes: 10

roxanahickey
roxanahickey

Reputation: 61

In case anyone is still searching for an answer (I was), there is a function to calculate this in the R package phyloseq: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0061217

I also found this tutorial useful: http://enterotype.embl.de/enterotypes.html

Upvotes: 6

Forzaa
Forzaa

Reputation: 1545

According to wikipedia Jensen-Shannon divergence is a transformation of the KL divergence. Applying the formula from the definition should give you the JS divergence then...

See: http://en.wikipedia.org/wiki/Jensen%E2%80%93Shannon_divergence

Upvotes: 0

Related Questions