CODEWITHSUNDEEP
pysparkapache-spark-sql
wrek
wrek

Reputation: 1161

Pyspark, TypeError: 'Column' object is not callable

How do I print out the content of a column of the doing the following operation? I am trying to print out the content of the column of the abcd, in the normal df, I can do df.show().

But how do I show the column objects?

>>> df = spark.createDataFrame([
...     ('a', 1.0, 1.0), ('a',1.0, 0.2), ('b', 1.0, 1.0),
...     ('c' ,1.0, 0.5), ('d', 0.55, 1.0),('e', 1.0, 1.0)
... ])
>>> df.show()
+---+----+---+                                                                  
| _1|  _2| _3|
+---+----+---+
|  a| 1.0|1.0|
|  a| 1.0|0.2|
|  b| 1.0|1.0|
|  c| 1.0|0.5|
|  d|0.55|1.0|
|  e| 1.0|1.0|
+---+----+---+

>>> abcd = spark_sum(
...     when(
...         spark_abs(
...             df['_2'] -
...             df['_3']
...         ) < 0.05,
...         "odt"
...     ).otherwise(0)
... ).alias(
...     'yo,man'
... )

how do I print out the content of abcd?

>>> abcd
Column<sum(CASE WHEN (abs((_2 - _3)) < 0.05) THEN odt ELSE 0 END) AS `yo,man`>

Upvotes: 1

Views: 7088

Answers (1)

user9767675
user9767675

Reputation:

how do I print out the content of abcd?

Pretty simple. Select and show:

df.select(abcd).show()

Upvotes: 2

Related Questions