CODEWITHSUNDEEP
scalaapache-sparkapache-spark-sql
hangc
hangc

Reputation: 5473

Joining columns when any match

I want to join two tables/dataframes based on some conditions.

For example in one table I have a column name

In the other table I have the columns name1, name2, name3

How do I join the tables on the columns, when any of the name columns in the second table matches, and use the columns in the order of name name2 name3 to join when possible?

Upvotes: 0

Views: 419

Answers (1)

Justin Pihony
Justin Pihony

Reputation: 67075

You can just use join that takes an expression as the conditional algorithm and use the or (||)

df1.join(df2, $"name" === $"name1" || $"name" === $"name2" || $"name" === $"name3")

Upvotes: 3

Related Questions