CODEWITHSUNDEEP
scalaapache-sparkdataframeapache-spark-sql
samba
samba

Reputation: 3101

scala - how to concatenate columns of a DataFrame with concat_ws?

I want to concatenate columns of my Dataframe. I wrote a Udf to achieve that but as I can see concat_ws expects columns while I'm passing it Array[String]. How should I pass columns of my DataFrame correctly?

val columns = salesDF.columns
val concatColumns = udf((arr: Seq[String]) => arr.mkString(" "))

salesDF.select(concat_ws(",", concatColumns(columns)))

Upvotes: 1

Views: 7837

Answers (1)

eliasah
eliasah

Reputation: 40370

You will just need to pass the selection as an Array[Column]:

import org.apache.spark.sql.functions.{col,concat_ws}

val selection = salesDF.columns.map(col)
salesDF.select(concat_ws(",", selection : _*))

Upvotes: 3

Related Questions