CODEWITHSUNDEEP
r
maximusdooku
maximusdooku

Reputation: 5512

How can divide a dataset based on percentage?

I have a dataset like this

ID      var value
9442000 a   2.01
9442000 v   2.2
9442000 h   5.3
9442000 f   0.2
9442000 s   0.55
9442000 t   0.6
952001  d   0.22
952001  g   0.44
952001  g   0.44
952001  h   0.77
652115  a   4.66
652115  d   1.55
652115  s   2.55
652115  s   2.55

I want to separate this into two dataframes for calibration (75%) and validation (25%). Doing it for overall is easy, but I want to do it ID-wise. So basically, I want to ensure that 75% of EACH ID goes to calibration. For example, for ID 9442000, I want to put any four events (random) into calibration and 2 into validation dataframe.

Expected output:

*Calibration*
 ID var value
9442000 a   2.01
9442000 v   2.2
9442000 h   5.3
9442000 f   0.2
952001  d   0.22
952001  g   0.44
952001  g   0.44
652115  a   4.66
652115  d   1.55
652115  s   2.55

And

*validation*
ID  var value
9442000 s   0.55
9442000 t   0.6
952001  h   0.77
652115  s   2.55

Upvotes: 2

Views: 2538

Answers (2)

IRTFM
IRTFM

Reputation: 263332

I thought you wanted a random sample and my original approach with ave went down in flames because the segments were too short, so I turned to tapply and unlist-ed the result

calib <- dat[ unlist( tapply(rownames(dat), dat$ID, 
                                        FUN=function(x) sample(x, .75*length(x)))), ]
valid <- dat[ !rownames(dat) %in% rownames(calib) , ]
#---------------
> str(calib)
'data.frame':   10 obs. of  3 variables:
 $ ID   : int  652115 652115 652115 952001 952001 952001 9442000 9442000 9442000 9442000
 $ var  : Factor w/ 8 levels "a","d","f","g",..: 1 6 2 4 2 4 7 8 1 6
 $ value: num  4.66 2.55 1.55 0.44 0.22 0.44 0.6 2.2 2.01 0.55
> str(valid)
'data.frame':   4 obs. of  3 variables:
 $ ID   : int  9442000 9442000 952001 652115
 $ var  : Factor w/ 8 levels "a","d","f","g",..: 5 3 5 6
 $ value: num  5.3 0.2 0.77 2.55
>

Upvotes: 3

Neal Fultz
Neal Fultz

Reputation: 9687

First, define a variable for which group it goes in, then use split:

> df$test <- ave(df$ID,df$ID,FUN=function(X) seq_along(X) %% 4 == 1  )
> 
> split(df, df$test)
$`0`
        ID var value test
2  9442000   v  2.20    0
3  9442000   h  5.30    0
4  9442000   f  0.20    0
6  9442000   t  0.60    0
8   952001   g  0.44    0
9   952001   g  0.44    0
10  952001   h  0.77    0
12  652115   d  1.55    0
13  652115   s  2.55    0
14  652115   s  2.55    0

$`1`
        ID var value test
1  9442000   a  2.01    1
5  9442000   s  0.55    1
7   952001   d  0.22    1
11  652115   a  4.66    1

Upvotes: 2

Related Questions