CODEWITHSUNDEEP
pythonpysparktuples
YAKOVM
YAKOVM

Reputation: 10153

StructType can not accept object float in pyspark

Why this is working just fine

from pyspark.sql.types import *
l=[("foo",83.33)]
schema = StructType([
   StructField("type", StringType(), True),
   StructField("value", DoubleType(), True)])
df= spark.createDataFrame(l,schema)

And this

 l=[(83.33)]
    schema = StructType([
    StructField("value", DoubleType(), True)])
    df= spark.createDataFrame(l,schema)

Gives me an error

StructType can not accept object 83.33 in type <class 'float'>

Upvotes: 2

Views: 9607

Answers (1)

Bala
Bala

Reputation: 11274

When using a one element tuple, a trailing comma is required. Check this TupleSyntax

>>> l=[(83.33,)]           //note the comma (,)
>>> schema = StructType([
...     StructField("value", DoubleType(), True)])
>>> df= spark.createDataFrame(l,schema)
>>> df.show()
+-----+
|value|
+-----+
|83.33|
+-----+

Upvotes: 7

Related Questions