CODEWITHSUNDEEP
scalaapache-sparkapache-spark-sql
EranM
EranM

Reputation: 303

How to perform join on MySQL (JDBC) with Spark?

I would like to read data from MySQL through Spark. The API which I saw is able to read data from specific table. something like,

val prop = new java.util.Properties
prop.setProperty("user", "<username>")
prop.setProperty("password", "<password>")

sparkSession.read.jdbc("jdbc:mysql://????:3306/???", "some-table", prop)

Now, I would like to perform a query for join tables. Does anyone know how to do it (on the database side, not with Spark SQL) ?

Thanks,

Eran

Upvotes: 2

Views: 2681

Answers (1)

user6022341
user6022341

Reputation:

You'll need to use the "table " argument as a query:

val table = "(SELECT foo JOIN bar ON foo.id = bar.id) as t"

spark.read.jdbc("jdbc:mysql://????:3306/???", table, prop)

You should note that giving an alias to your query is important or this won't work.

Upvotes: 5

Related Questions