CODEWITHSUNDEEP
rsubsetdescribe
TSpinelli
TSpinelli

Reputation: 21

R: Dropping variables using number of observations

I have a large dataset, and I'm trying to drop some of my variables based on how many observations each has. For instance, I would like to drop any variable in my dataframe where n < 3 (total observations for that variable is less than 3). Since R can count observations for each variable using describe, can't I use that number to subset the data instead of having to type in each variable name each time I pull in a new version (each version has different variables that will have low n's and there are over 40 variables). Thanks so much for your help!

For instance, my data looks like this:

ID    Runaway    Aggressive   Emergency  Hospitalization   Injury
1      3            NA            4          1              NA
2      NA           NA            2          1              NA
3      4            NA            6          2              3
4      1            NA            1          1              NA

I want to be able to drop "Aggressive" and "Injury" based on their n's being 0 and 1 respectively. However, instead of telling R to drop them by variable name, it would be much more convenient if it was possible to tell R to drop any variable where n < 3 (or whatever number I choose) as I'll be using this code for multiple versions of this dataset. I have tried using column numbers (which is better than writing them out) but it's still pretty tedious when I have to describe() the data, figure out which variables have low n's, and then drop 28 variables or subset() around them. This works but it's cumbersome...

UIRCorrelation <- UIRKidUnique61[c(28, 30, 32, 34:38, 42, 54:74)]

For some reason, my example looks different when I'm editing versus when I save so I also included an image of it. Sorry. This is the first time I've ever used stack overflow to ask a question. I actually spent a lot of time googling this but couldn't find an answer relating to n.

This line did not work: DF[, sapply(DF, function(col) length(na.omit(col))) > 4]

Upvotes: 1

Views: 781

Answers (2)

TSpinelli
TSpinelli

Reputation: 21

This function did the trick: 

valid <- function(x) {sum(!is.na(x))}

N <- apply(UIRCorrelation,2,valid) 

UIRCorrelation2 <- UIRCorrelation[N > 3]

Upvotes: 0

Andre Elrico
Andre Elrico

Reputation: 11480

DF being your dataframe

DF[, sapply(DF, function(col) length(na.omit(col))) > 4]

Upvotes: 2

Related Questions