Is there a way to store outputs in a for_loop function?

Question

I have to compute model accuracy across a range of values. I would need to create a for loop to feed in these range of values, and then store these outputs in an empty dataframe.

I tried to use my python experience to create an empty list initially, and tried to iterate the for loop across a range of the values.

Train_Rest_Loop Dataframe:

  ground Rest_Cos_Sim
1      1         0.25
2      1         0.20
3      1         0.35
4      1         0.25
5      1         0.25
6      0         0.30


cos_sim_values <- seq(0,1, by=0.05)
Accuracy <- matrix(NA, nrow=21, ncol=2) # Empty Matrix

for (i in cos_sim_values) {
train_rest_loop['Rest_Cos_Sim'] <- ifelse(train_rest_loop$Rest_Cos_Sim >= i, 1,0)
cm_train_rest <- table(train_rest_loop$ground, train_rest_loop$Rest_Cos_Sim)
cm_train_rest <- caret :: confusionMatrix(cm_train_rest, mode = "prec_recall", positive="1") 
Accuracy[i,] <- as.vector(cm_train_rest$overall['Accuracy']) }

Error in !all.equal(nrow(data), ncol(data)) : invalid argument type

No actual results because the code won't run, see error message above.

I would expect a list of accuracy values:

  cos_sim_values Accuracy
1      0.1          0.25
2      0.15         0.20
3      0.20         0.35
4      0.25         0.25
5      0.30         0.25
6      0.40         0.30

Answer 1

One way using lapply instead of a loop and tryCatch in case there is an error is the following.
I have used as.integer(condition), not ifelse, since the outcome is a binary 1 or 0.

library(caret)
library(e1071)

cos_sim_values <- seq(0, 1, by = 0.05)

result_list <- lapply(cos_sim_values, function(csvals){
  trl[['Rest_Cos_Sim']] <- as.integer(train_rest_loop$Rest_Cos_Sim >= csvals)
  trl[['Rest_Cos_Sim']] <- factor(trl[['Rest_Cos_Sim']], levels = 0:1)
  cm_train_rest <- table(trl)
  cm_train_rest <- tryCatch(confusionMatrix(cm_train_rest, mode = "prec_recall", positive = "1"),
                            error = function(e) e)
})

ok <- !sapply(result_list, inherits, "error")
result_list[ok]

Accuracy <- sapply(result_list[ok], '[[', 'overall')[1, ]
Accuracy
# [1] 0.8333333 0.8333333 0.8333333 0.8333333 0.8333333
# [6] 0.6666667 0.3333333 0.1666667 0.1666667 0.1666667
#[11] 0.1666667 0.1666667 0.1666667 0.1666667 0.1666667
#[16] 0.1666667 0.1666667 0.1666667 0.1666667 0.1666667
#[21] 0.1666667

Data.

train_rest_loop <- read.table(text = "
  ground Rest_Cos_Sim
1      1         0.25
2      1         0.20
3      1         0.35
4      1         0.25
5      1         0.25
6      0         0.30
", header = TRUE)

Answer 2

The error message comes form the line

cm_train_rest <- caret :: confusionMatrix(cm_train_rest, mode = "prec_recall", positive="1")

and says, that cm_train_rest should have the same number of columns and rows what is not the case.

To your main question:

Accuracy[i,] <- as.vector(cm_train_rest$overall['Accuracy'])

will also not work as i holds in you case the values of cos_sim_values which could not be used as an index.

In addition in

train_rest_loop['Rest_Cos_Sim'] <- ifelse(train_rest_loop$Rest_Cos_Sim >= i, 1,0)

you are overwriting the column Rest_Cos_Sim what you might not want.

Maybe the following brings you closer to your expected result:

library("caret")
library("e1071")

cos_sim_values <- seq(0,1, by=0.05)
Accuracy <- matrix(NA, nrow=length(cos_sim_values), ncol=2) # Empty Matrix

train_rest_loop <- data.frame(ground=factor(c(1,1,1,1,1,0)), Rest_Cos_Sim=c(0.25,0.20,0.35,0.25,0.25,0.30))

for (idx in 1:length(cos_sim_values)) { #use idx as an index which ranges from 1 to the length of cos_sim_values
  i <- cos_sim_values[idx]  #Get the idx'th values of cos_sim_values
  tt <- factor(ifelse(train_rest_loop$Rest_Cos_Sim >= i, 1,0), levels = c(0,1))
  cm_train_rest <- table(train_rest_loop$ground, tt)
  cm_train_rest <- caret :: confusionMatrix(cm_train_rest, mode = "prec_recall", positive="1") 
  Accuracy[idx,] <- c(i, as.vector(cm_train_rest$overall['Accuracy']))
}