CODEWITHSUNDEEP
r
Julien Navarre
Julien Navarre

Reputation: 7840

Generate unique alphanumeric IDs

I have a data frame and I want to add to it a column that contains not duplicated alphanumeric values.

Firstly, I adapted a function that I found on a blog. (https://ryouready.wordpress.com/2008/12/18/generate-random-string-name/)

idGenerator <- function(n, lengthId) {

  alphaNum <- c(0:9, letters, LETTERS)

  if (n > length(alphaNum)^lengthId) {
    return("Error! n > perms : Infinite loop")
  }

  idList <- rep(NULL, n)

  for (i in 1:n) {
    idList[i] <- paste(sample(alphaNum, 
                                    lengthId, replace = TRUE), collapse = "")
    while(idList[i] %in% idList[-i]) {
      idList[i] <- paste(sample(alphaNum, 
                                lengthId, replace = TRUE), collapse = "")
    }
  }

  return(idList)
}

My problem is that my dataframe has about 250k rows so with n = 250k this function is just running for ever. I know that with n = 250k, if I increase the length of the id string (lengthId) the odds to get the same string are unrealistic so the while loop is such a waste of ressources but I really need to be sure that will not happen, I mean "sure" with control structures.

So I found a more efficient way to do it, instead of calling a while and checking all the vector for each i in the loop, I check if there is duplicated in the final vector :

idGenerator <- function(n, lengthId) {

  alphaNum <- c(0:9, letters, LETTERS)

  if (n > length(alphaNum)^lengthId) {
    return("Error! n > perms : Infinite loop")
  }

  idList <- 1:n

  for (i in 1:n) {
    idList[i] <- paste(sample(alphaNum, 
                              lengthId, replace = TRUE), collapse = "")
  }

  while(any(duplicated(idList))) {
    idList[which(duplicated(idList))] <- paste(sample(alphaNum, lengthId, 
                                                replace = TRUE), collapse = "")
  }

  return(idList)
}

It's very slow if the while must run a lot of times => When n is very close to the number of permutations.

> system.time(idGenerator(62^2, 2))
    utilisateur     système     écoulé 
    8.00            0.00        8.02 

 > system.time(idGenerator(62^3, 3))

 Timing stopped at: 584.35 16.66 602.46

But it's quite acceptable for a long id string :

> system.time(idGenerator(250000, 12))
    utilisateur     système     écoulé 
    3.2             0.0         3.2

However it's still 3sec+ to create a column so I'm looking for a faster way. I know that the loop isn't so good and I should prefer vectorization but I'm not realy a master of code optimization. So if you have any ideas, thank you in advance.

Upvotes: 3

Views: 5773

Answers (1)

A5C1D2H2I1M1N2O1R2T1
A5C1D2H2I1M1N2O1R2T1

Reputation: 193637

I would suggest looking at the stri_rand_strings function from the "stringi" package:

library(stringi)
stri_rand_strings(10, 3)
 # [1] "wsm" "FvH" "UXm" "14t" "rvv" "Pfo" "mzK" "20b" "O9P" "ZOr"
system.time(X <- stri_rand_strings(250000, 12))
#    user  system elapsed 
#   0.327   0.003   0.333 
length(unique(X))
# [1] 250000
head(X)
# [1] "WxRPZjt0uFaI" "E129Ug0Vif3f" "qXGzQDO0LzvG" 
# [4] "9D4guGMf2jZ1" "Qw1p7reH4XKg" "0gziFNnZ16p8"

Upvotes: 14

Related Questions