CODEWITHSUNDEEP
pythonpyspark
Codegator
Codegator

Reputation: 637

Pyspark: Fill a fix value in pyspark column if other column is null

I have a pyspark dataframe which has two columns. I want to populate one column with a fix value if row value in other column is null. So in customer_df if customer_address is null then populate city column as 'unknown'

I am trying this

customer_df = customer_df.withColumn('city',when(customer_df.customer_address == '','unknown')

But this gives syntax error. What is that I am missing here? Thanks in advance

Upvotes: 0

Views: 98

Answers (1)

Lamanus
Lamanus

Reputation: 13551

customer_df = customer_df.withColumn('city', 
    when(col(customer_address).isNull(), 'unknown').otherwise(col('city'))
  )

Upvotes: 3

Related Questions