CODEWITHSUNDEEP
raucmlrmulticlass-classification
Roshana
Roshana

Reputation: 367

how to use multiclass.au1p measure in mlr

I am trying to use multiclass.au1p measure in mlr package. It gave me an error saying

Error in FUN(X[[i]], ...) : Measure multiclass.au1p requires predict type to be: 'prob'!

When I tried to set the predict type to prob then it gave me an error similar to following for any classifier i used

Error in setPredictType.Learner(learner, predict.type) : Trying to predict probs, but classif.xgboost.multiclass does not support that!

How can I resolve this?

Following is my code

  trainTask <- makeClassifTask(data = no_out_pso,target = "response_grade")

  Clslearn = makeLearner("classif.xgboost", predict.type = "prob")
  Clslearn = makeMulticlassWrapper(Clslearn, mcw.method = "onevsrest")
  Clslearn = setPredictType(Clslearn, "prob")

  rdesc = makeResampleDesc("CV", iters = 3)

  r = resample(Clslearn, trainTask, rdesc, measures = list(mlr::acc, mlr::multiclass.au1p, mlr::multiclass.au1u))

  print(r)

Upvotes: 0

Views: 575

Answers (2)

PhilippPro
PhilippPro

Reputation: 698

It does not work with the makeMulticlassWrapper, because this does not support probability prediction (at the moment). I get also an error, when I try to set it to prob in your code.

Code that works: 

Clslearn = makeLearner("classif.xgboost", predict.type = "prob")
rdesc = makeResampleDesc("CV", iters = 3)
r = resample(Clslearn, iris.task, rdesc, measures = list(mlr::acc, mlr::multiclass.au1p, mlr::multiclass.au1u))

Upvotes: 4

Lars Kotthoff
Lars Kotthoff

Reputation: 109242

You need to use a classifier that supports predicting probabilities. You can get a list with the listLearners() function:

listLearners(properties = "prob")

Upvotes: 2

Related Questions