CODEWITHSUNDEEP
javascalaapache-sparkapache-spark-sql
Shankar
Shankar

Reputation: 8957

Spark DataFrame - select list of columns using Java API

I'm trying to select List of columns from the DataFrame using Java API.

Sample Java Code:

List<String> colList = Arrays.asList(new String[] { "column1", "column2", "column3" });
df.selectExpr((String[])colList.toArray()).show();

In Java API, I have to use selectExpr instead of select. Is there any other way of selecting list of columns using Java API.

But in Scala, I can do something like below.

Sample Scala Code:

val colList = List("column1", "column2", "column3")
df.select(colList.head, colList.tail: _*).show

Upvotes: 1

Views: 8536

Answers (1)

Alper t. Turker
Alper t. Turker

Reputation: 35219

You can use array of String:

String[] colList =  { "column1", "column2", "column3" };
String first = colList[0];
String[] rest =  Arrays.copyOfRange(colList, 1, colList.length);

logData.select(first, rest);

or array of Column:

import static org.apache.spark.sql.functions.col;
import org.apache.spark.sql.Column;

Column[] colList =  { col("column1"), col("column2"), col("column3") };
logData.select(colList);

Upvotes: 13

Related Questions