CODEWITHSUNDEEP
scalaapache-sparkapache-spark-mllibapache-spark-ml
Vinoth Manamala
Vinoth Manamala

Reputation: 9

How to return a value to a val using if statement?

I am trying to convert a var assignment to a val assignment. Currently my code is

  // Numerical vectorizing for normalization
  var normNumericalColNameArray: Array[String] = Array()
  if (!continousPredictors.sameElements(Array(""))) {
    if (paramStandardize) {
      println("Apply standardization")
      stages += new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)
      stages += new StandardScaler()
        .setWithMean(true)
        .setWithStd(true)
        .setInputCol(NumericalFeaturesCol)
        .setOutputCol(StandardizedNumericalFeaturesCol)
      normNumericalColNameArray = Array(StandardizedNumericalFeaturesCol)
    } else {
      println("Not apply standardization")
      stages += new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)
      normNumericalColNameArray = Array(NumericalFeaturesCol)
    }
  }

  stages += new VectorAssembler().setInputCols(normNumericalColNameArray ++ oneHotCatColNamesArray).setOutputCol(FeaturesCol)

and I want to do something like this

  
  val normNumericalColNameArray =
    if (continousPredictors.nonEmpty && paramStandardize) {
    println("Apply standardization")
    stages += new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)
    stages += new StandardScaler()
      .setWithMean(true)
      .setWithStd(true)
      .setInputCol(NumericalFeaturesCol)
      .setOutputCol(StandardizedNumericalFeaturesCol)
    Array(StandardizedNumericalFeaturesCol)
    } else if (continousPredictors.nonEmpty){
    println("Not apply standardization")
    stages += new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)
    Array(NumericalFeaturesCol)
    }


  stages += new VectorAssembler().setInputCols(normNumericalColNameArray ++ oneHotCatColNamesArray).setOutputCol(FeaturesCol)

and I run into this error

value ++ is not a member of Any
stages += new VectorAssembler().setInputCols(normNumericalColNameArray ++ oneHotCatColNamesArray).setOutputCol(FeaturesCol)

I am trying to return the Array from the if condition into my Val normNumericalColNameArray. Can somebody please help?

Upvotes: 0

Views: 58

Answers (3)

Levi Ramsey
Levi Ramsey

Reputation: 20561

For an if expression, the resulting type is the least-upper-bound of the branches. If there's no final else, there's an implicit

else ()  // where () is the singleton value of the Unit type

So in your if {} else if without a final else, the overall expression will have the type of whatever the least-upper-bound of Array[String], Array[String], and Unit is, which is Any, which then becomes the type inferred for normNumericalColNameArray.

As Tim and gifa note, you can fix this by adding a final else clause. There's also no need to hoist the inner if into the outer condition check:

val normNumericalColNameArray =
  if (continousPredictors.nonEmpty) {
    stages += new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)

    if (paramStandardize) {
      println("Apply standardization")
      stages += new StandardScaler()
        .setWithMean(true)
        .setWithStd(true)
        .setInputCol(NumericalFeaturesCol)
        .setOutputCol(StandardizedNumericalFeaturesCol)
      Array(StandardizedNumericalFeaturesCol)
    } else {
      println("Not apply standardization")
      Array(NumericalFeaturesCol)
    }
  } else {
    Array[String]()
  }

Upvotes: 1

gifa
gifa

Reputation: 86

The problem is in the if statement. You have:

val normNumericalColNameArray = {
  if (condition1){
   Array[String]()
  }
  else if (condition2){
   Array[String]()
  }
}

Now, your variable has type Any because if condition1 is not true and condition2 is not true, then your if statement (function) doesn't return anything: you actually made a partial function. Try adding a final else statement, like:

val normNumericalColNameArray = {
  if (condition1){
   Array[String]()
  }
  else if (condition2){
   Array[String]()
  }
  else{
   Array[String]()
  }
}

It will work then. Also, I would recommend to make your normNumericalColNameArray a lazy val, so to be evaluated only when it is required, since you are modifying stages in the if statement.

Upvotes: 0

Tim
Tim

Reputation: 27421

The problem is that there is an if without an else block, so the result could be nothing (Unit). Just add an else clause:

} else if (continousPredictors.nonEmpty) {
  println("Not apply standardization")
  stages += new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)

  Array(NumericalFeaturesCol)
} else {
  Array()
}

But Array is a Java type, so prefer to use Vector or List if possible.

Upvotes: 1

Related Questions