CODEWITHSUNDEEP
python-3.xapache-sparkpysparkapache-spark-sqlsql-order-by
Jack
Jack

Reputation: 997

Pyspark dataframe OrderBy list of columns

I am trying to use OrderBy function in pyspark dataframe before I write into csv but I am not sure to use OrderBy functions if I have a list of columns.

Code:

Cols = ['col1','col2','col3']
df = df.OrderBy(cols,ascending=False)

Upvotes: 18

Views: 49964

Answers (1)

user9921104
user9921104

Reputation:

As per docstring / signature:

Signature: df.orderBy(*cols, **kwargs)
Docstring:
Returns a new :class:`DataFrame` sorted by the specified column(s).
:param cols: list of :class:`Column` or column names to sort by.
:param ascending: boolean or list of boolean (default True).

Both 

df = spark.createDataFrame([(1, 2, 3)] )
cols = ["_1", "_2", "_3"]

df.orderBy(cols, ascending=False)

and 

df.orderBy(*cols, ascending=False)

are valid, as well as equivalents with list[pyspark.sql.Column].

Upvotes: 26

Related Questions