Reputation: 871
The mlr function configureMlr() allows users to set the following parameter:
on.learner.error: What should happen if an error in an underlying learning algorithm is caught
“warn”: A FailureModel will be created, which predicts only NAs and a warning will be generated.
What is the best way to check if a FailureModel has been returned? At the moment I am just checking the class of the model, and if it is not what it should be then I am assuming it is a FailureModel.
library(survival)
library(mlr)
library(mlrCPO)
data(veteran)
set.seed(24601)
task_id = "MAS"
mas.task <- makeSurvTask(id = task_id, data = veteran, target = c("time", "status"))
mas.task <- createDummyFeatures(mas.task)
preproc_pipeline <- cpoScale() # Standardise the numerical data - center and scale
outer = makeResampleDesc("CV", iters=5, stratify=TRUE) # Benchmarking
cox.lrn <- preproc_pipeline %>>% makeLearner(cl="surv.coxph", id = "coxph", predict.type="response")
learners = list( cox.lrn )
bmr = benchmark(learners=learners, tasks=mas.task, resamplings=outer, measures=list(cindex), show.info = TRUE, models=TRUE)
model_id = 'coxph.scale'
mods = getBMRModels(bmr, learner.ids = c(model_id))
num_models = length(mods[[task_id]][[model_id]])
for (i in 1:num_models) {
mod = getLearnerModel(mods[[task_id]][[model_id]][[i]], more.unwrap=TRUE)
if (class(mod) == "coxph") {
print(mod$coefficients)
} else {
print("Failure model")
}
}
I tried the following,
if (isFailureModel(mod)) {
print("FailureModel")
}
but got the error message:
Error in UseMethod("isFailureModel") :
no applicable method for 'isFailureModel' applied to an object of class "coxph"
Upvotes: 1
Views: 42
Reputation: 6302
I don't think there is an easy solution to this (as least I am not aware of it).
Your approach does not seem to be far off from succeeding. However, as stated in ?mlr::isFailureModel()
, it needs to be applied to a an object of class WrapperModel
, not to an an object of a specific model class (e.g. coxph in your case).
Upvotes: 2