CODEWITHSUNDEEP
rduplicatesuniquecolumnsorting
AlexP
AlexP

Reputation: 147

How do I remove duplicate values within a column throughout the dataset in R

The dataframe looks like the following:

        var_1 var_2 var_3
        5     2     2
        5     2     1
        6     3     4

dput of data

data <- structure(list(var_1 = c(5L, 5L, 6L), var_2 = c(2L, 2L, 3L), 
  var_3 = c(2L, 1L, 4L)), row.names = c(NA, -3L), class = "data.frame")

How do I remove duplicate values within a column, and repeat this for all columns so the data looks like:

       var_1 var_2 var_3
       5     2     2
       6     3     1
                   4

Upvotes: 1

Views: 186

Answers (3)

Yuriy Saraykin
Yuriy Saraykin

Reputation: 8880

base

data <- structure(list(var_1 = c(5L, 5L, 6L), var_2 = c(2L, 2L, 3L), 
                       var_3 = c(2L, 1L, 4L)), row.names = c(NA, -3L), class = "data.frame")




tmp <- sapply(data, unique)

res <- sapply(tmp, "length<-", max(lengths(tmp)))
res
#>      var_1 var_2 var_3
#> [1,]     5     2     2
#> [2,]     6     3     1
#> [3,]    NA    NA     4
colSums(res, na.rm = T)
#> var_1 var_2 var_3 
#>    11     5     7

Created on 2021-04-15 by the reprex package (v2.0.0)

Upvotes: 0

Sinh Nguyen
Sinh Nguyen

Reputation: 4497

A try return a list with different length after unique. In data.frame you always need them to have equal length with NA filled in

library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library(purrr)
data <- structure(list(var_1 = c(5L, 5L, 6L), var_2 = c(2L, 2L, 3L), 
  var_3 = c(2L, 1L, 4L)), row.names = c(NA, -3L), class = "data.frame")

# get unique lsit
map(data, unique)
#> $var_1
#> [1] 5 6
#> 
#> $var_2
#> [1] 2 3
#> 
#> $var_3
#> [1] 2 1 4

# get unique & sum in one take
map(data, function(x) { sum(unique(x)) })
#> $var_1
#> [1] 11
#> 
#> $var_2
#> [1] 5
#> 
#> $var_3
#> [1] 7

Shorter version with summarize if only need to calculate the sum of unique value

# if all you need is the sum of unique value then you can just do this
data %>%
  summarize(across(.fns = function(x) { sum(unique(x)) }))
#>   var_1 var_2 var_3
#> 1    11     5     7

Created on 2021-04-15 by the reprex package (v2.0.0) Created on 2021-04-15 by the reprex package (v2.0.0)

Upvotes: 0

Jon Spring
Jon Spring

Reputation: 66775

Here's one approach, pivoting longer, filtering for distinct column-value combinations, then spreading:

library(tidyverse)
df %>%
  pivot_longer(cols = everything()) %>%
  distinct(name, value) %>%
  group_by(name) %>%
  mutate(row = row_number()) %>%
  ungroup() %>%
  pivot_wider(names_from = name, values_from = value)

Result

# A tibble: 3 x 4
    row var_1 var_2 var_3
  <int> <int> <int> <int>
1     1     5     2     2
2     2     6     3     1
3     3    NA    NA     4

If you want the sum of distinct numbers, you could add as a last line:

%>% summarize(across(-row, ~sum(., na.rm = TRUE)))

Result:

# A tibble: 1 x 3
  var_1 var_2 var_3
  <int> <int> <int>
1    11     5     7

Upvotes: 2

Related Questions