CODEWITHSUNDEEP
apache-spark-sqlpyspark
isha
isha

Reputation: 153

Pyspark error passing StructType to Schema

Getting Error : TypeError: 'StructType' object is not callable, while passing StructType to the schema method. Below is the code:

final_schema = StructType([StructField("id", StringType(), True)])
dataframe = sc.read.text('/path').schema(final_schema)

Data is string type as below:

id
AO_01
AO_02
AO_03

Upvotes: 4

Views: 11761

Answers (1)

isha
isha

Reputation: 153

Changed the call to sc.read.schema(final_schema).text('/path') because text(...) will return a dataframe instance and need to set schema before that.

Upvotes: 6

Related Questions