CODEWITHSUNDEEP
apache-sparkapache-spark-sqlpyspark
Sophie Dinka
Sophie Dinka

Reputation: 83

AttributeError: 'NoneType' object has no attribute 'write in Pyspark

Spark: 2.4.4 Pyspark

I have registered temp table and trying to save output to a csv file. but getting error as "AttributeError: 'NoneType' object has no attribute 'write'"

data.registerTempTable("data")
output = spark.sql("SELECT col1,col2,col3 FROM data").show(truncate = False)

output.write.format('.csv').save("D:/BPR-spark/sourcefile/filtered.csv")

please help

Upvotes: 3

Views: 9750

Answers (1)

PRZU
PRZU

Reputation: 76

You are assigning the result of show() to the variable output and show() doesn't return a value.

So, you want to assign the Dataframe to the variable output, and then saving it like this:

data.registerTempTable("data")
output = spark.sql("SELECT col1,col2,col3 FROM data")
output.write.format('.csv').save("D:/BPR-spark/sourcefile/filtered.csv")

Upvotes: 6

Related Questions