CODEWITHSUNDEEP
rsubsetr-caretdata-partitioning
happy_sisyphus
happy_sisyphus

Reputation: 1827

How does createDataPartition function from caret package split data?

From the documentation:

For bootstrap samples, simple random sampling is used.

For other data splitting, the random sampling is done within the levels of y when y is a factor in an attempt to balance the class distributions within the splits.

For numeric y, the sample is split into groups sections based on percentiles and sampling is done within these subgroups.

For createDataPartition, the number of percentiles is set via the groups argument.

I don't understand why this "balance" thing is needed. I think I understand it superficially, but any additional insight would be really helpful.

Upvotes: 6

Views: 26547

Answers (1)

loki
loki

Reputation: 10350

It means, if you have a data set ds with 10000 rows 

set.seed(42)
ds <- data.frame(values = runif(10000))

with 2 "classes" with unequal distribution (9000 vs 1000)

ds$class <- c(rep(1, 9000), rep(2, 1000))
ds$class <- as.factor(ds$class)
table(ds$class)
#    1    2 
# 9000 1000

you can create a sample, which tries to maintain the ratio / "balance" of the factor classes. 

dpart <- createDataPartition(ds$class, p = 0.1, list = F)
dsDP <- ds[dpart, ]
table(dsDP$class)
#   1   2 
# 900 100

Upvotes: 17

Related Questions