CODEWITHSUNDEEP
pythonapache-sparkpysparkapache-spark-sql
Sid
Sid

Reputation: 91

Spark Sql query works with hardcoded value but not with variable

I have a PySpark program in which there is a SQL query like this:

id = 'abc1'

info = sqlc.sql("SELECT * from df WHERE isn = 'id'")

This query does not work. I am returned with a blank set of values. However, if I hardcode the value, it works.

info = sqlc.sql("SELECT * from df WHERE isn = 'abc1'")

I have tried str(id) just to be sure. How can I resolve this?

Upvotes: 0

Views: 1468

Answers (1)

Alper t. Turker
Alper t. Turker

Reputation: 35229

Use format in Python 3.5 or before:

info = sqlc.sql("SELECT * from df WHERE isn = '{id}'".format(id=id))

or

info = sqlc.sql("SELECT * from df WHERE isn = '{}'".format(id))

Literal string formatting in Python 3.6 and later:

info = sqlc.sql(f"SELECT * from df WHERE isn = '{id}'")

Upvotes: 2

Related Questions