CODEWITHSUNDEEP
apache-sparkpysparkapache-spark-sql
user1450410
user1450410

Reputation: 301

pyspark use range between function in spark sql

when i run 

spark.sql(''' 
        select client,avg(amount) over (partition by client
         order by my_timestamp
         range between interval 30 days preceding and current row) as moving_avg_amount
         from my_table''')

it works.

but if i want to exclude the last X days - it fails.

... range between interval 30 days preceding and 12 days preceding ..
   or :
 ... range between interval 30 days preceding and interval 12 days preceding ..

what is the correct syntax for that?

Upvotes: 0

Views: 1102

Answers (1)

RickyG
RickyG

Reputation: 138

In Spark SQL Window over interval of between two specified time boundaries - between 3 hours and 2 hours ago it is suggested that range intervals do not work properly in SparkSQL.

Upvotes: 1

Related Questions