CODEWITHSUNDEEP
rbroom
Doug Fir
Doug Fir

Reputation: 21242

Why does broom::augment return more rows than input data?

I noticed that while using Broom's augment function, the newly created data frame has more rows than I originally started with. e.g.

# Statistical Modeling
## dummy vars
library(tidyverse)
training_data <- mtcars
dummy <- caret::dummyVars(~ ., data = training_data, fullRank = T, sep = ".")
training_data <- predict(dummy, mtcars) %>% as.data.frame()
clean_names <- names(training_data) %>% str_replace_all(" |`", "")
names(training_data) <- clean_names

## make target a factor
target <- training_data$mpg
target <- ifelse(target < 20, 0,1) %>% as.factor() %>% make.names()

## custom evaluation metric function
my_summary  <- function(data, lev = NULL, model = NULL){
  a1 <- defaultSummary(data, lev, model)
  b1 <- twoClassSummary(data, lev, model)
  c1 <- prSummary(data, lev, model)
  out <- c(a1, b1, c1)
  out}

## tuning & parameters
set.seed(123)
train_control <- trainControl(
  method = "cv",
  number = 3,
  sampling = "up", # over sample due to inbalanced data
  savePredictions = TRUE,
  verboseIter = TRUE,
  classProbs = TRUE,
  summaryFunction = my_summary
)

linear_model = train(
  x = select(training_data, -mpg), 
  y = target,
  trControl = train_control,
  method = "glm", # logistic regression
  family = "binomial",
  metric = "AUC"
)

library(broom)
linear_augment <- augment(linear_model$finalModel)

Now, if I look at my new augmented data frame and compare to the original mtcars one:

> nrow(mtcars)
[1] 32
> nrow(linear_augment)
[1] 36

Expectation was for 32 rows not 36. Why is that?

Upvotes: 0

Views: 92

Answers (1)

zack
zack

Reputation: 5405

You're upsampling in your trainControl call, resulting in more samples than your original data set.

## tuning & parameters
set.seed(123)
train_control <- trainControl(
  method = "cv",
  number = 3,
  # sampling = "up", # over sample due to inbalanced data
  savePredictions = TRUE,
  verboseIter = TRUE,
  classProbs = TRUE,
  summaryFunction = my_summary
)

linear_model = train(
  x = select(training_data, -mpg), 
  y = target,
  trControl = train_control,
  method = "glm", # logistic regression
  family = "binomial",
  metric = "AUC"
)
library(broom)
linear_augment <- augment(linear_model$finalModel)

Note upsampling is commented out

> dim(linear_augment)
[1] 32 19

Upvotes: 2

Related Questions