CODEWITHSUNDEEP
javaapache-sparkapache-spark-sql
lte__
lte__

Reputation: 7576

Spark SQL - JAVA syntax of CASE-THEN?

I noticed I can use CASE-THEN with Spark if I use an SQLContext and the .sql() function. Is there a way to use this in a JAVA syntax, directly on dataframes, too? How? Now, I write:

SparkConf sparkConf = new SparkConf();
JavaSparkContext ctx = new JavaSparkContext(sparkConf);
SQLContext sqlContext = new SQLContext(ctx);
DataFrame df = //some imported data 
df.registerTempTable("df");
sqlContext.sql("SELECT *use case-then in here* FROM df");

I'm looking for something like

df.select(case("this").then("that"));

Upvotes: 5

Views: 5722

Answers (1)

Yuan JI
Yuan JI

Reputation: 2995

Just import org.apache.spark.sql.functions then use when(Column col, Object obj).

import org.apache.spark.sql.functions;

df.select(functions.when(df.col("colName").equalTo("this"), "that").otherwise("something"));

Upvotes: 8

Related Questions