CODEWITHSUNDEEP
pythondataframeapache-sparkpysparkapache-spark-sql
Rahul Diggi
Rahul Diggi

Reputation: 378

How to create a new column of datatype map<string,string> in pyspark

I want to create a new column(not from an existing columns) having MapType(StringType(),StringType()) how do I achieve this.

So far, I was able to write the below code but it's not working.

df = df.withColumn("normalizedvariation",lit(None).cast(MapType(StringType(),StringType())))

Also I would like to know the different methods to achieve the same, Thank you.

Upvotes: 1

Views: 793

Answers (1)

Jayalekshmi R J
Jayalekshmi R J

Reputation: 81

This is one way you can try, newMapColumn here woukld be the name of the Map column. You can see the output I got below. If that is not what you are looking for, please let me know. Thanks!

Also you will have to import the functions using the below line:
from pyspark.sql.functions import col,lit,create_map

enter image description here

Upvotes: 2

Related Questions