CODEWITHSUNDEEP
rr-mice
Yue Harriet Huang
Yue Harriet Huang

Reputation: 4179

Imputation MICE in R still NA left in dataset

After running MICE package, the number of missing values are shrinked from 147428 to 46093 in each of the 5 complete imputation sets. But isn't it supposed to be 0 NAs instead???

Thanks!

Here is my MICE code:

imp = mice(newdata)

imputationSet1 = complete(imp)
imputationSet2 = complete(imp,2)
imputationSet3 = complete(imp,3)
imputationSet4 = complete(imp,4)
imputationSet5 = complete(imp,5)

Upvotes: 10

Views: 15687

Answers (4)

Phil
Phil

Reputation: 21

Try to handover an additional parameter called threshold, whose default is 0.999. If you set this to something closer to 1 or even larger one, your problem should disappear.

Be aware though that this issue arises only if the collinearity in the data is high.

Upvotes: 2

lara
lara

Reputation: 176

Ben, the mice() function detects multicollinearity, and solves the prob- lem by removing one or more predictors for the matrix. Each removal is noted in the loggedEvents element of the mids object. For example,

imp <- mice(cbind(nhanes, chl2 = 2 * nhanes$chl), print = FALSE)

imp$loggedEvents

informs us that the duplicate variable chl2 was removed before iteration. The algorithm also detects multicollinearity during iterations.

Another measure to control the algorithm is the ridge parameter. The ridge parameter is specified as an argument to mice(). Setting ridge=0.001 or ridge=0.01 makes the algorithm more robust at the expense of bias.

At the terminal node, we can apply a simple method like mice.impute.sample() that does not need any predictors for itself.

This information is taken from the book Flexible Imputation of Missing Data by Stef van Buuren, p. 129

Upvotes: 10

lara
lara

Reputation: 176

What helped me is to convert as-character variables to as-factor variables and NAs have disappeared from the imputed dataset.

Upvotes: 5

Ben Ogorek
Ben Ogorek

Reputation: 597

Yeah there should be no missing values left.

I bet there are some rows in your data set that are so badly mangled with missingness that mice's imputation models break down. Is it possible that there are rows in your dataset where every value is missing? That would do it.

Another thing to try on a whim - crank up the number of iterations to 15: imp = mice(newdata, maxit = 15). Does that change anything?

Upvotes: 0

Related Questions