CODEWITHSUNDEEP
dataframepysparkapache-spark-sqljupyter-notebookapache-toree
Gaurav Tambi
Gaurav Tambi

Reputation: 33

How to show a dataframe in a good format in jupyter notebook using pyspark with apachee-toree kernel

It will become very difficult to read the output if it is not in the good formated manner like shown in the image in this link -> required output. I am running my code on jupyter notebook in pyspark with apache-toree kernel. Currently when i show my dataframe it looks somewhat like as shown in the image in this link -> Current Table.

I know there is a way using "magic" as mentioned on apache toree website but it is also not working in pyspark. It works if i load a normal python notebook in jupyter. It gives the error shown in the image with pyspark

I am not able to find anything on internet about this. Any help is very much appreciated.

Upvotes: 1

Views: 672

Answers (1)

AFault
AFault

Reputation: 9

I work around this problem by converting pyspark DataFrame to pandas DataFrame with toPandas method.

Upvotes: 0

Related Questions