Reputation: 161
I am trying to compute a weighted kappa with confidence intervals in R.
I am having trouble understanding why the two functions, DescTools::CohenKappa and irr::kappa2, give different outputs. I need to use the former (DescTools::CohenKappa) as I need the confidence intervals, but would like to know why these produce different results?
DescTools::CohenKappa:
kappa lwr.ci upr.ci
-0.24584718 -0.45930367 -0.03239068
irr::kappa2:
Cohen's Kappa for 2 Raters (Weights: 0,0.25,1)
Subjects = 20
Raters = 2
Kappa = 0.378
z = 1.71
p-value = 0.0878
Here is an example:
library(irr)
library(DescTools)
## Data
test = structure(list(fi.cat = structure(c(2L, 2L, 2L, 2L, 2L, 1L, 1L,
2L, 3L, 2L, 1L, 2L, 1L, 1L,
3L, 3L, 2L, 2L, 1L, 1L), .Label = c("non-frail","prefrail", "frail"), class = "factor"),
fp.cat = structure(c(2L,2L, 2L, 2L, 3L, 1L, 2L,
1L, 3L, 2L, 1L, 2L, 1L, 1L,
2L, 3L, 2L, 3L, 1L, 1L), .Label = c("non-frail", "prefrail", "frail"), class = "factor")),
row.names = c(NA,20L), class = "data.frame")
## DescTools::CohenKappa
m = table(test$fi.cat, test$fp.cat) # create confusion matrix (i.e. cross tabulation)
wm = matrix(c(
0, 0.25, 1,
0.25, 0, 0.25,
1, 0.25, 0), ncol=3, byrow=TRUE # my weights
)
kappa1 = DescTools::CohenKappa(m, weights=wm, conf.level=0.95)
## irr::kapp2
kappa2 = irr::kappa2(test[, c("fi.cat", "fp.cat")], c(0, 0.25, 1))
Upvotes: 1
Views: 314