CODEWITHSUNDEEP
python-3.xdataframepyspark
Mr.Bean
Mr.Bean

Reputation: 77

How to convert str to list in Pyspark dataframe?

My input is

enter image description here

In pandas dataframe I used literal_eval to convert 'hashtags' column from str to list, but I don't know how to do that in Spark Dataframe. Any solution for my case?

Tks all supports for my case!

Upvotes: 0

Views: 179

Answers (1)

vilalabinot
vilalabinot

Reputation: 1611

You can use:

df1.withColumn("list_hashtags", expr("from_json(hashtags, 'array<string>')"))

Therefore, list_hashtags[0] will give you #ucl.

Hope this is what you want!

Upvotes: 1

Related Questions