CODEWITHSUNDEEP
rfunctionloopsdataframena
user8840683
user8840683

Reputation:

Removing all empty columns and rows in data.frame when rows don't go away

I have two data.frames, dA (HERE) and dB (HERE). They are exactly the same EXCEPT that dB has one completely empty column and multiple empty rows.

dA <- read.csv("https://raw.githubusercontent.com/izeh/m/master/irr3.csv", h = T)
dB <- read.csv("https://raw.githubusercontent.com/izeh/m/master/irr4.csv", h = T)

I want remove all empty columns and all empty rows in dB so that dB becomes exactly like dA.

Currently, I try the following to achieve my goal but it looks like empty rows don't get removed: 

# remove columns with all NA
B1 <- dB[, colSums(is.na(dB)) != nrow(dB)]

# remove rows with all NA
B2 <- B1[rowSums(is.na(B1)) != ncol(B1), ]   # NOW, check by:  `nrow(B2)` the `NA` haven't 
                                             # been removed !!

Upvotes: 2

Views: 2107

Answers (3)

GKi
GKi

Reputation: 39667

You can use replace to convert "" to NA and use then is.na. To remove complete rows and colls you can use apply with the function all.

x  <- is.na(replace(dB, dB=="", NA))
dB[!apply(x, 1, all), !apply(x, 2, all)]

or using rowSums and colSums, like in the question:

x <- is.na(dB) | dB == ""
dB[rowSums(x) != ncol(x), colSums(x) != nrow(x)]

Upvotes: 1

akrun
akrun

Reputation: 887193

Here is an option with Filter from base R

Filter(function(x) !all(is.na(x)), dB)
#  study.name  group.name outcome ESL prof scope type
#1  Shin.Ellis    ME.short       1   1    2     1    1
#2  Shin.Ellis     ME.long       1   1    2     1    1
#3  Shin.Ellis   DCF.short       1   1    2     1    2
#4  Shin.Ellis    DCF.long       1   1    2     1    2
#5  Shin.Ellis  Cont.short       1   1    2    NA   NA
#6  Shin.Ellis   Cont.long       1   1    2    NA   NA
#7                              NA  NA   NA    NA   NA
#8    Trus.Hsu       Exper       1   2    2     2    1
#...

Or with any

Filter(function(x) any(!is.na(x)), dB)

To remove the rows

B1[!!rowSums(!is.na(B1) & B1 != ""),]

Or using Reduce

B1[Reduce(`|`, lapply(B1, function(x) !is.na(x) & x != "" )),]

Upvotes: 1

Ronak Shah
Ronak Shah

Reputation: 388992

You have NA and also empty rows. You can do

B1[rowSums(is.na(B1) | B1 == "") != ncol(B1), ]

#   study.name  group.name outcome ESL prof scope type
#1  Shin.Ellis    ME.short       1   1    2     1    1
#2  Shin.Ellis     ME.long       1   1    2     1    1
#3  Shin.Ellis   DCF.short       1   1    2     1    2
#4  Shin.Ellis    DCF.long       1   1    2     1    2
#5  Shin.Ellis  Cont.short       1   1    2    NA   NA
#6  Shin.Ellis   Cont.long       1   1    2    NA   NA
#8    Trus.Hsu       Exper       1   2    2     2    1
#.....

We can also use filter_all from dplyr

library(dplyr)
B1 %>% filter_all(any_vars(!is.na(.) & . != ""))

Upvotes: 1

Related Questions