CODEWITHSUNDEEP
dataframepyspark
Aravind P
Aravind P

Reputation: 65

How to convert a dataframe to a variable

Is there any direct function to convert a dataframe and assign to a variable? For example below returns this

>>> partitionRecordCount= spark.sql("select count(*) from mydb.mytable where partition_date="yyyymmdd")
>>> partitionRecordCount.show()

+--------+
|count(1)|
+--------+
|  206157|
+--------+

what i need is like below

>>> partitionRecordCount
206157

I need that record count integer value directly in that variable on the left hand side rather than a dataframe. Please advice

Upvotes: 2

Views: 1877

Answers (2)

Hillygoose
Hillygoose

Reputation: 195

See this answer get value out of dataframe

So for your example you can just change it to:

partitionRecordCount = partitionRecordCount.collect()[0]

Upvotes: 1

Rahul
Rahul

Reputation: 767

Try

partitionRecordCount.collect()[0][0]

Upvotes: 3

Related Questions