CODEWITHSUNDEEP
rperformancedata.tablenormalization
Stan
Stan

Reputation: 1237

Normalize each row of data.table

This seems like it should be easy, but I can't find an answer :(. I'm trying to normalize each row of a data_table like this:

normalize <- function(x) {
  s = sum(x)
  if (s>0) {
    return(x/s)
  } else {
    return 0
  }
}

How do I call this function on every row of a data.table and get a normalized data.table back? I can do a for loop, but that's surely not the right way, and apply(data, 1, normalize) will, as I understand, convert my data.table to a matrix which will be a big performance hit.

Upvotes: 4

Views: 4067

Answers (4)

Frank
Frank

Reputation: 66819

Here's one way to avoid coercing to a matrix:

cols = names(DT)
DT[, s := Reduce("+",.SD)]
DT[s > 0, (cols) := lapply(.SD,"/",s), .SDcols = cols]
DT[s <=  0, (cols) := 0]
DT[, s := NULL]

This is what I would do if there was a good reason to use a data.table over a matrix (in a later step).

Upvotes: 4

David Arenburg
David Arenburg

Reputation: 92282

Considering this example data set (next time, please provide an example data set yourself)

set.seed(123) 
DT <- data.table(x = rnorm(10), y = rnorm(10), z = rnorm(10))

I would try avoiding by row operations and vecotrize using rowSums, something like the following

DT[, names(DT) := {temp = rowSums(.SD) ; (.SD / temp) * (temp > 0)}]
DT
#              x          y          z
#  1:  0.0000000  0.0000000  0.0000000
#  2:  0.0000000  0.0000000  0.0000000
#  3:  1.6697906  0.4293327 -1.0991233
#  4:  0.0000000  0.0000000  0.0000000
#  5:  0.0000000  0.0000000  0.0000000
#  6:  0.9447911  0.9843707 -0.9291618
#  7:  0.2565558  0.2771142  0.4663301
#  8:  0.0000000  0.0000000  0.0000000
#  9:  0.0000000  0.0000000  0.0000000
# 10: -1.3289000 -1.4097961  3.7386962

The reason I've created temp is in order to avoid running rowSums(.SD) twice. The *(temp > 0) part is basically your if and else statement. It returns a logical vector of TRUE/FALSE which then converted to 1/0 and then multiplied against (.SD/temp)

Upvotes: 8

giraffehere
giraffehere

Reputation: 1148

There's probably an easier (and faster) way to do this with apply, but this method works. I think it's more readable too, but that's just my opinion.

# Creating sample data.
myDF <- data.frame(a = seq(1, 50), b = seq(1, 100, 2) , c = seq(1, 200, 4))
# Going through each row and dividing its contents by the sum of that row.
for (row in rownames(myDF)) { myDF[row, ] <- myDF[row, ] / sum(myDF[row, ]) }

Note, this does require your rownames to be numbers, though.

Upvotes: 0

Chris Watson
Chris Watson

Reputation: 1367

Here's what I came up with. First you need to edit your function (I believe), so that it returns rep(0, length(x)) instead of just 0.

set.seed(123); DT <- data.table(x=rnorm(1e3), y=rnorm(1e3), z=rnorm(1e3))
> DT
                x           y          z
   1: -0.56047565 -0.99579872 -0.5116037
   2: -0.23017749 -1.03995504  0.2369379
   3:  1.55870831 -0.01798024 -0.5415892
   4:  0.07050839 -0.13217513  1.2192276
   5:  0.12928774 -2.54934277  0.1741359
  ---                                   
 996: -0.08997520  0.07664366  1.0609662
 997:  1.07051604  0.25516476 -0.4455056
 998: -1.35110039  0.27744682 -0.4291802
 999: -0.52261670  0.53685602  1.1890118
1000: -0.24919068 -0.46048557  0.8342941
> DT[, c('x', 'y', 'z') := as.list(normalize(c(x, y, z))), by=1:nrow(DT)]                                                                             
> DT
                x           y          z
   1:  0.00000000  0.00000000  0.0000000
   2:  0.00000000  0.00000000  0.0000000
   3:  1.56005167 -0.01799574 -0.5420559
   4:  0.06091117 -0.11418417  1.0532730
   5:  0.00000000  0.00000000  0.0000000
  ---                                   
 996: -0.08588413  0.07315877  1.0127254
 997:  1.21625341  0.28990225 -0.5061557
 998:  0.00000000  0.00000000  0.0000000
 999: -0.43433718  0.44617122  0.9881660
1000: -1.99963905 -3.69518205  6.6948211

Upvotes: 0

Related Questions