CODEWITHSUNDEEP
xgboost
Chandra
Chandra

Reputation: 524

XGBoost - custom loss function

There are two different guidelines on using customized loss function in xgboost.

If predicted probability ‘p’ = sigmoid(z)

  1. In https://github.com/dmlc/xgboost/blob/master/demo/guide-python/custom_objective.py 1, line#25 mentions that gradient of customized loss function should be taken w.r.t 'z’

2 . In https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html 1, gradient is w.r.t 'p’

Which is correct?

Upvotes: 1

Views: 2677

Answers (1)

gazza89
gazza89

Reputation: 151

To keep this as general as possible, you need to calculate the gradient of the total loss function w.r.t changing the current predicted values. Normally, your loss function will be of the form $L = \sum_{i=1}^{N} \ell (y_{i}, \hat{y_{i}})$, in which $y_{i}$ is the label of the $i^{th}$ datapoint and $\hat{y_{i}}$ is your prediction (in the binary classification case, you might choose to define it such that $y_{i}$ are the binary labels, and $\hat{y_{i}}$ are the probabilities the classifier assigns to the label being one of the classes).

You then need to calculate $\frac{\partial\ell}{\hat{y_{i}}}\big|{y{i}}$

Upvotes: 1

Related Questions