CODEWITHSUNDEEP
scalasortingapache-sparkdataframeapache-spark-sql
Praveen Agrawal
Praveen Agrawal

Reputation: 39

How to sort Array[Row] by given column index in Scala

How to sort Array[Row] by given column index in Scala?

I'm using RDD[Row].collect() which gives me array[Row], but I want to sort it based on a given column index.

I have already used quick-sort logic and it's working, but there are too many for loops and all.

I would like to use a Scala built-in API which can do this task with the minimum amount of code.

Upvotes: 0

Views: 2008

Answers (1)

Tzach Zohar
Tzach Zohar

Reputation: 37832

It would be much more efficient to sort the Dataframe before collecting it - if you collect it, you lose the distributed (and parallel) computation. You can use Dataframe's sort, for example - ascending order by column "col1":

val sorted = dataframe.sort(asc("col1"))

Upvotes: 1

Related Questions