Reputation: 1
I'm trying to implement a custom quantile loss function using LightGBM in R, which works like the already implemented quantile loss, to later add further customisation to this function. However, when I train models for different quantiles, the predictions are the same for all quantiles.
This is my code:
library(lightgbm)
library(MASS)
# Define custom quantile loss with gradient and hessian
custom_quantile_loss <- function(preds, dtrain, alpha) {
labels <- getinfo(dtrain, "label")
residuals <- preds - labels
# Gradient calculation based on quantile level
grad <- ifelse(residuals < 0, (1 - alpha), -alpha)
# Hessian is constant (for stability, we set it to 1.0)
hess <- rep(1.0, length(residuals))
return(list(grad = grad, hess = hess))
}
# Custom LightGBM model training function
train_custom_quantile_lgb <- function(data, label, quantiles, num_iterations = 500,
learning_rate = 0.05, max_depth = 7) {
# Create quantile-specific datasets
dtrain_list <- lapply(quantiles, function(alpha) {
dtrain <- lgb.Dataset(data = data, label = label)
return(dtrain)
})
predictions <- matrix(NA, nrow = nrow(data), ncol = length(quantiles))
for (i in seq_along(quantiles)) {
alpha <- quantiles[i]
# Define model parameters with monotonicity constraint
params <- list(
objective = "regression",
metric = "none",
learning_rate = learning_rate,
max_depth = max_depth
)
# Train the model with custom quantile loss function
model <- lgb.train(
params = params,
data = dtrain_list[[i]],
nrounds = num_iterations,
obj = function(preds, dtrain) custom_quantile_loss(preds, dtrain, alpha),
verbose = -1
)
# Make predictions for the current quantile
predictions[, i] <- predict(model, data)
}
colnames(predictions) <- paste0("q_", quantiles)
return(predictions)
}
# Example usage
set.seed(123)
data(iris)
train_data <- as.matrix(iris[, 2:4])
label <- as.numeric(iris$Sepal.Length)
quantiles <- c(0.1, 0.5, 0.9)
# Train model and predict for quantiles 0.1, 0.5, and 0.9
quantile_predictions <- train_custom_quantile_lgb(data = train_data,
label = label, quantiles = quantiles)
# Output the first rows of the predictions
head(quantile_predictions)
which results in
q_0.1 q_0.5 q_0.9
[1,] 5.205249 5.205249 5.205249
[2,] 4.628195 4.628195 4.628195
[3,] 4.629957 4.629957 4.629957
[4,] 4.727256 4.727256 4.727256
[5,] 5.205249 5.205249 5.205249
[6,] 5.503609 5.503609 5.503609
What am I doing wrong?
Upvotes: 0
Views: 47