CODEWITHSUNDEEP
sqlsql-serverapache-sparkpysparkapache-spark-sql
user3331874
user3331874

Reputation: 47

Facing issue while writing SQL in pyspark

I am trying to convert below SQL code to pyspark. Can someone please help me

UPDATE B 
SET col = 'a'
FROM table B
WHERE (case when left(util,1) in ('A', 'B') then count  else PROCS end < 1)

Here, util, count, procs are column names.

while coding in pyspark, i can create a new column 'col' like this:

df1 = df1.withColumn("col", case condition ,a)

Upvotes: 0

Views: 48

Answers (1)

mck
mck

Reputation: 42402

You can use when for doing the equivalent of update:

df1 = df1.withColumn(
    'col', 
    F.when(
        F.expr("case when left(util,1) in ('A', 'B') then count else PROCS end < 1"),
        F.lit('a')
    ).otherwise(F.col('col'))
)

Upvotes: 1

Related Questions