CODEWITHSUNDEEP
sqlapache-sparkpysparkjvmsnowflake-cloud-data-platform
pdangelo4
pdangelo4

Reputation: 230

spark snowflake connector sql query SQL compilation error

I'm trying to query snowflake using input variable from my function (run via aws glue).. Below are 3 things I've tried but I keep getting the following error:

An error occurred while calling o14310.load. SQL compilation error: error line 1 at position 94

Any ideas what the issue may be? Regular queries not using an input variable runs fine..

def s_counts(df, s_id):
what_to_query = "select R_ID FROM mytable WHERE S_ID ="+s_id
CurrentCount = spark.read.format(SNOWFLAKE_SOURCE_NAME)
   .options(**sfOptions)
   .option("query", what_to_query)
   .load()

CurrentCount = spark.read.format(SNOWFLAKE_SOURCE_NAME)
   .options(**sfOptions)
   .option("query", "select R_ID FROM mytable WHERE S_ID = s_id")
   .load()

CurrentCount = spark.read.format(SNOWFLAKE_SOURCE_NAME)
   .options(**sfOptions)
   .option("query", "select R_ID FROM mytable WHERE S_ID = "+s_id)
   .load()

Upvotes: 0

Views: 929

Answers (1)

ggordon
ggordon

Reputation: 10035

Try wrapping s_id in quotes. For example

what_to_query = "select R_ID FROM mytable WHERE S_ID = '{0}'".format(s_id)

or

what_to_query = "select R_ID FROM mytable WHERE S_ID = '%s'" % (s_id)

Let me know if this works for you.

Upvotes: 1

Related Questions