CODEWITHSUNDEEP
javaapache-sparkapache-spark-sql
Boris
Boris

Reputation: 453

How to use Column.isin in Java?

I'm trying to filter a Spark DataFrame using a list in Java.

java.util.List<Long> selected = ....;
DataFrame result = df.filter(df.col("something").isin(????));

The problem is that isin(...) method accepts Scala Seq or varargs.

Passing in JavaConversions.asScalaBuffer(selected) doesn't work either.

Any ideas?

Upvotes: 8

Views: 10607

Answers (2)

Popeye
Popeye

Reputation: 338

A bit shorter version would be:

df.filter(col("something").isin(selected.toArray()));

Upvotes: 2

Shankar
Shankar

Reputation: 8957

Use stream method as follows:

df.filter(col("something").isin(selected.stream().toArray(String[]::new))))

Upvotes: 13

Related Questions