CODEWITHSUNDEEP
pyspark
ERJAN
ERJAN

Reputation: 24500

how to create dataframe from one column in pyspark?

I have sliced out one column of type Column in pyspark.

x =game_reviews.groupBy("product_id_index").agg((F.count('star_rating').alias('num')  ) 
x.num

gives

Column<b'num'>

But this 

new_df = spark.createDataFrame(x.num)
new_df.show()

gives error.

Upvotes: 0

Views: 76

Answers (1)

michalrudko
michalrudko

Reputation: 1530

What you want to achieve is a simple one-liner. Good luck!

new_df = game_reviews.groupBy("product_id_index").agg((F.count('star_rating').alias('num')).select("num")
new_df.show()

Upvotes: 1

Related Questions