qweurtz
qweurtz

Reputation: 1

Custom Quantile Loss Function in LightGBM R Predicts Same Value for All Quantiles

I'm trying to implement a custom quantile loss function using LightGBM in R, which works like the already implemented quantile loss, to later add further customisation to this function. However, when I train models for different quantiles, the predictions are the same for all quantiles.

This is my code:

    library(lightgbm)
    library(MASS)

    # Define custom quantile loss with gradient and hessian
    custom_quantile_loss <- function(preds, dtrain, alpha) {
      labels <- getinfo(dtrain, "label")
      residuals <- preds - labels
  
      # Gradient calculation based on quantile level
      grad <- ifelse(residuals < 0, (1 - alpha), -alpha)
  
      # Hessian is constant (for stability, we set it to 1.0)
      hess <- rep(1.0, length(residuals))
  
      return(list(grad = grad, hess = hess))
    }

    # Custom LightGBM model training function
    train_custom_quantile_lgb <- function(data, label, quantiles, num_iterations = 500, 
            learning_rate = 0.05, max_depth = 7) {
  
      # Create quantile-specific datasets
      dtrain_list <- lapply(quantiles, function(alpha) {
        dtrain <- lgb.Dataset(data = data, label = label)
        return(dtrain)
      })
  
      predictions <- matrix(NA, nrow = nrow(data), ncol = length(quantiles))
  
      for (i in seq_along(quantiles)) {
        alpha <- quantiles[i]
    
        # Define model parameters with monotonicity constraint
        params <- list(
          objective = "regression",
          metric = "none",
          learning_rate = learning_rate,
          max_depth = max_depth
        )
    
        # Train the model with custom quantile loss function
        model <- lgb.train(
          params = params,
          data = dtrain_list[[i]],
          nrounds = num_iterations,
          obj = function(preds, dtrain) custom_quantile_loss(preds, dtrain, alpha),
          verbose = -1
        )
    
        # Make predictions for the current quantile
        predictions[, i] <- predict(model, data)
      }
  
      colnames(predictions) <- paste0("q_", quantiles)
      return(predictions)
    }

    # Example usage
    set.seed(123)
    data(iris)
    train_data <- as.matrix(iris[, 2:4])
    label <- as.numeric(iris$Sepal.Length)
    quantiles <- c(0.1, 0.5, 0.9)

    # Train model and predict for quantiles 0.1, 0.5, and 0.9
    quantile_predictions <- train_custom_quantile_lgb(data = train_data, 
    label = label, quantiles =   quantiles)

    # Output the first rows of the predictions
    head(quantile_predictions)

which results in

            q_0.1    q_0.5    q_0.9
    [1,] 5.205249 5.205249 5.205249
    [2,] 4.628195 4.628195 4.628195
    [3,] 4.629957 4.629957 4.629957
    [4,] 4.727256 4.727256 4.727256
    [5,] 5.205249 5.205249 5.205249
    [6,] 5.503609 5.503609 5.503609

What am I doing wrong?

Upvotes: 0

Views: 47

Answers (0)

Related Questions