CODEWITHSUNDEEP
pysparkpivottranspose
Asma Damani
Asma Damani

Reputation: 197

How to transpose specific columns in PySpark

I have a following PySpark dataframe:

year    week    date    time          value
2020    1     20201203  2:00 - 2:15    23.9
2020    1     20201203  2:15 - 2:30    45.87
2020    1     20201203  2:30 - 2:45    87.76
2020    1     20201203  2:45 - 3:00    12.87

I want to transpose the time and value column. The desired output should be:

 year   week    date    2:00 - 2:15     2:15 - 2:30     2:30 - 2:45    2:45 - 3:00
 2020   1     20201203    23.9             45.87           87.76          12.87

Upvotes: 0

Views: 87

Answers (1)

Emma
Emma

Reputation: 9363

You can use groupby and pivot.

df = df.groupby('year', 'week', 'date').pivot('time').max('value')

Upvotes: 2

Related Questions