CODEWITHSUNDEEP
apache-sparkpyspark
Moein
Moein

Reputation: 1730

How to order fields in a spark row object?

Once we create a row object in pyspark, the fields in the row will be alphabetically ordered:

>> my_row = Row(rate=0.1, height=1)
>> print(my_row)
>> Row(height=1, rate=0.1)

How can I have the fields ordered in the way I want, say: Row(rate=0.1, height=1)

Upvotes: 1

Views: 322

Answers (1)

Moein
Moein

Reputation: 1730

the simplest way (no need to create a dataframe) to create a row with a 'custom' field order is as follows:

fields = ["rate", "height"] # in our desired order
values = {"height": 1, "rate": 0.1}

my_row_class = Row(*fields) # create a row class first
my_row = my_row_class(*[values[key] for key in fields])

>> print my_row
>> Row(rate=0.1, height=1)

Upvotes: 2

Related Questions