CODEWITHSUNDEEP
rentropy
Annika Magnusson
Annika Magnusson

Reputation: 45

entropy measure poLCA Mplus

I realised that entropy measures for latent class by MPLus and poLCA (R package) refers different range of values.

In MPLUS,, the value ranges between 0 and 1. They mention entropy measure indicates class overlap, and if number gets closer to 1, the classes are well-separated, or vice versa.

In R, entropy measure is higher than 1, such as 5, 12.

Do you know if any conversion is possible between the two measures? How can I calculate entropy measure in R between 0 and 1?

Upvotes: 1

Views: 2931

Answers (2)

Israel Souza
Israel Souza

Reputation: 43

Mplus calculate relative entropy (Celeux, G. and Soromenho, G. (1996) An Entropy Criterion for Assessing the Number of Clusters in a Mixture Model. Journal of Classification, 13, 195-212. http://dx.doi.org/10.1007/BF01246098).

For this, in R I use

data(carcinoma)
f <- cbind(A,B,C,D,E,F,G)~1
lca2 <- poLCA(f,carcinoma,nclass=2)
##RELATIVE ENTROPY
##Numerator:
nume.E <- -sum(lca2$posterior * log(lca2$posterior))
##Denominator (n*log(K)): ## n is a sample size, and K is a number of class
deno.E <- 118*log(2)
##Relative Entropy
Entro <- 1-(nume.E/deno.E)
Entro

Another way for this (an course by Daniel Oberski)

entropy<-function (p) sum(-p*log(p))
error_prior <- entropy(lca2$P) # Class proportions
error_post <- mean(apply(lca2$posterior, 1, entropy))
R2_entropy <- (error_prior - error_post) / error_prior
R2_entropy

Upvotes: 3

Jebediah15
Jebediah15

Reputation: 794

I am also trying to figure this out. Are you using the poLCA.entropy function from the poLCA package? The manual states that it "calculates the entropy of a cross-classification table produced as a density estimate using a latent class model." I wonder if the value could be standardized to the 0-1 range using the maximum value in the range that R calculates (logarithm of the total number of cells in the fitted cross-classfication table), but I haven't been able to find guidance on this.

Upvotes: 0

Related Questions