CODEWITHSUNDEEP
scalaapache-spark
Chaouki
Chaouki

Reputation: 465

How to filter a row if the value contains in list in scala spark?

I want to filter a spark dataframe using list, for example

var lisst=List(1,2,34)
df.filter(col("id).isin(lisst))

give the error

Unsupported literal type classscala.collection.immutable.$colon$colon 
List(1,2,34)

I tried with Seq and Set and get same error.

Upvotes: 2

Views: 9775

Answers (2)

Steven Black
Steven Black

Reputation: 2232

you've got to expand the list :) 

import org.apache.spark.sql.functions._

var lisst=List(1,2,34)
df.filter(col("id).isin(lisst:_*)).show()

Upvotes: 0

koiralo
koiralo

Reputation: 23109

You can use isin function as below

var lisst=List(1,2,34)
 df.filter(col("id").isin(lisst :_*))

Hope this helps!

Upvotes: 8

Related Questions