Pyspark: adding parameter to a filter function

Question

Please, I would appreciate some help: I need to add a parameter to this function but this syntax is not correct:

val input_parameter = 100
df_filtered = (df.filter(( (F.col('LOCID') != F.col('LOCID2') ) & 
                           (F.col('distance') <= F.col('RADIUSINMETERS') + F.col('RADIUSINMETERS2') + input_parameter)  
                        ))
              )

Many thanks!

Answer 1

Spark needs to know that you are passing a variable, in order to do that we need to ensure we are passing within lit() function , from the documentation

Creates a Column of literal value.

more read here