CODEWITHSUNDEEP
rdata.tablesample
Farrel
Farrel

Reputation: 10444

How do you extract a few random rows from a data.table on the fly

I have a large data.table (about 24000 rows and growing). I want to subset that datatable based on a couple of criteria and from that subset (ends up being about 3000 rows) I want to randomly sample just 4 rows. I do not want to create a named 3000 or so row data.table, count its rows and then sample based on row number. How can I do it on the fly? Or should I just suck it up by creating the table and then working on it, sampling it and then using rm() to get rid of it?

Lets simulate my issue

require(data.table)
random.length  <-  sample(x = 15:30, size = 1)
data.table(city=sample(c("Cape Town", "New York", "Pittsburgh", "Tel Aviv", "Amsterdam"), size=random.length, replace = TRUE), score = sample(x=1:10, size = random.length, replace=TRUE))

That makes a random length table, which simulates the fact that depending on my criteria and depending on my starting table, I do not know what the length of the subsetted table with be

Now, if I just wanted the first three rows I could do as so

data.table(city=sample(c("Cape Town", "New York", "Pittsburgh", "Tel Aviv", "Amsterdam"), size=random.length, replace = TRUE), score = sample(x=1:10, size = random.length, replace=TRUE))[1:3]

But let us say I did not want the first three rows but rather a random 3 rows, then I would want to do something such as this... 

data.table(city=sample(c("Cape Town", "New York", "Pittsburgh", "Tel Aviv", "Amsterdam"), size=random.length, replace = TRUE), score = sample(x=1:10, size = random.length, replace=TRUE))[sample(x= 1:number of rows of that previous data.table,size = 3 ]

That will not work. How do I compute, on the fly, what the length of the initial data.frame was?

Upvotes: 33

Views: 30253

Answers (3)

Daniel
Daniel

Reputation: 1262

Another alternative way would be to use sapply approach.
For example: 

  as.data.table(sapply(DT[], sample, 10))

Upvotes: 1

Matt Dowle
Matt Dowle

Reputation: 59592

Have just made .N work in i. New README item :

.N is now available in i, FR#724. Thanks to newbie indirectly here and Farrel directly here.

This now works :

DT[...][...][sample(.N,3)]

e.g.

> random.length  <-  sample(x = 15:30, size = 1)
> data.table(city = sample(c("Cape Town", "New York", "Pittsburgh", "Tel Aviv", "Amsterdam"),size=random.length, replace = TRUE), score = sample(x=1:10, size = random.length, replace=TRUE))[sample(.N, 3)] 
         city score
1:   New York     4
2: Pittsburgh     3
3:  Cape Town     9
>

Upvotes: 67

djhurio
djhurio

Reputation: 5536

There is a two step approach:

  1. Compute the index i using .I
  2. Sample on index i

Example code.

require(data.table)
random.length  <-  sample(x = 15:30, size = 1)
data.table(city = sample(c("Cape Town", "New York", "Pittsburgh", "Tel Aviv", "Amsterdam"),size=random.length, replace = TRUE), score = sample(x=1:10, size = random.length, replace=TRUE))[,i := .I][sample(i, 3)]

Upvotes: 3

Related Questions