CODEWITHSUNDEEP
apache-spark
Martin Moore
Martin Moore

Reputation: 59

How to filter a dataset rows using variables

I try to filter rows of a dataset using variables like so :

Dataset<Row> dataset = dF.select(dF.col("*")).filter(col(list.get(0)) == lit(list.get(1))));

But i get a compilation error :

Cannot resolve filter(boolean)

What's the solution to this ?

Upvotes: 1

Views: 38

Answers (1)

Vincent Doba
Vincent Doba

Reputation: 5078

Filter takes column instead of boolean as parameter. So to compare columns you should use equalTo method that will return a column instead of ==:

Dataset<Row> dataset = dF.select(dF.col("*")).filter(col(list.get(0)).equalTo(lit(list.get(1)))));

Upvotes: 1

Related Questions