CODEWITHSUNDEEP
r
Reactormonk
Reactormonk

Reputation: 21730

grouped loess scores

I've got this data.frame: http://sprunge.us/TMGS, and I'd like to calculate the loess of Intermediate.MAP.Score ~ x, so I get one curve from the whole dataset. But every group (by name) should have the same wight as every other group, I'm not sure what happens if I call loess over the whole data.frame. Do I need to call it once per group and combine them? If yes, how do I do that?

Upvotes: 0

Views: 959

Answers (1)

IRTFM
IRTFM

Reputation: 263471

If you want to average over all of the values in 'loess.fits' produced in my earlier answer to a different question, you will get one answer. If you want to just get a loess fit on the entire dataset (which would not fit your "equal weighting" spec at least as I interpret that phrase), you will get another answer.

This would produce averaged 'yhat' values at the 51 equally spaced data values for 'x' in the range of [0,1]. Because of missing values, it may not be exactly "equally weighted" but only at the extremes. The estimates are dense elsewhere:

apply( as.data.frame(loess.fits), 1, mean, na.rm=TRUE)

enter image description here

Earlier answer: I would have titled the question "loess scores split by group":

plot(dat$x, dat$Intermediate.MAP.Score, col=as.numeric(factor(dat$name)) )

If you proceed with loess(Intermediate.MAP.Score ~ x, data=dat) you will get an overall average with no distinction among groups. And loess doesn't accept factor or character arguments in its formula. You need to split by 'name' and calculate separately. The other gotcha to avoid is plotting on the default limits which will be driven varying data ranges:

loess.fits <- lapply(split(dat, dat$name), function(xdf) { 
              list( yhat=predict( loess(Intermediate.MAP.Score ~ x, 
                                    data=xdf[ complete.cases(
                                            xdf[ , c("Intermediate.MAP.Score", "x") ]
                                                               ), 
                                                 ] ) ,
                                  newdata=data.frame(x=seq(0,1,by=0.02))))})
plot(dat$x, dat$Intermediate.MAP.Score, 
                   col=as.numeric(factor(dat$name)), 
                   ylim=c(0.2,1) )
lapply(loess.fits, function(xdf) { par(new=TRUE); 
                                 # so the plots can be compared to predictions
                                  plot(x= seq(0,1,by=0.02), y=xdf$yhat, 
                                       ylab="", xlab="",
                                       ylim=c(0.2,1), axes=FALSE) })

enter image description here

Upvotes: 2

Related Questions