CODEWITHSUNDEEP
rdataframedplyrmagrittr
Saqib Ali
Saqib Ali

Reputation: 4428

Removing rows in a R data.table with NAs in specific columns

I have a data.table with a large number of features. I would like to remove the rows where the values are NAs only for certain features.

Currently I am using the following to handle this:

data.joined.sample <- data.joined.sample  %>% 
  filter(!is.na(lat))   %>% 
  filter(!is.na(long))   %>% 
  filter(!is.na(temp))   %>% 
  filter(!is.na(year))   %>% 
  filter(!is.na(month))   %>% 
  filter(!is.na(day))   %>% 
  filter(!is.na(hour))   %>% 
.......

Is there a more concise way to achieve this?

str(data.joined.sample)
Classes ‘data.table’ and 'data.frame':  336776 obs. of  50 variables:

Upvotes: 1

Views: 1359

Answers (2)

akrun
akrun

Reputation: 886968

We can select those columns, get a logical vector of NA's based on it using complete.cases and use that to remove the NA elements

data.joined.sample[complete.cases(data.joined.sample[colsofinterest]),]

where 

colsofinterest <- c("lat", "long", "temp", "year", "month", "day", "hour")

Update

Based on the OP's comments, if it is a data.table, then subset the colsofinterest and use complete.cases

data.joined.sample[complete.cases(data.joined.sample[, colsofinterest, with = FALSE])]

Upvotes: 3

IRTFM
IRTFM

Reputation: 263311

data.table-objects, if that is in fact what your working with, have a somewhat different syntax for the "[" function. Look through this console session:

> DT = data.table(x=rep(c("b","a","c"),each=3), y=c(1,3,6), v=1:9)
> DT[x=="a"&y==1]
   x y v
1: a 1 4
> is.na(DT[x=="a"&y==1]$v) <- TRUE # make one item NA
> DT[x=="a"&y==1]
   x y  v
1: a 1 NA
> DT
   x y  v
1: b 1  1
2: b 3  2
3: b 6  3
4: a 1 NA
5: a 3  5
6: a 6  6
7: c 1  7
8: c 3  8
9: c 6  9
> DT[complete.cases(DT)]  # note no comma
   x y v
1: b 1 1
2: b 3 2
3: b 6 3
4: a 3 5
5: a 6 6
6: c 1 7
7: c 3 8
8: c 6 9
>  DT   # But that didn't remove the NA, it only gave a value
   x y  v
1: b 1  1
2: b 3  2
3: b 6  3
4: a 1 NA
5: a 3  5
6: a 6  6
7: c 1  7
8: c 3  8
9: c 6  9
> DT <- DT[complete.cases(DT)]  # do this assignment to make permanent
> DT
   x y v
1: b 1 1
2: b 3 2
3: b 6 3
4: a 3 5
5: a 6 6
6: c 1 7
7: c 3 8
8: c 6 9

Probably not the true "data.table way".

Upvotes: 2

Related Questions