CODEWITHSUNDEEP
pythonpyspark
mommomonthewind
mommomonthewind

Reputation: 4640

How to convert a list of array to Spark dataframe

Suppose I have a list:

x = [[1,10],[2,14],[3,17]]

I want to convert x to a Spark dataframe with two columns id (1,2,3) and value (10,14,17).

How could I do that?

Thanks

Upvotes: 3

Views: 19707

Answers (2)

Rahul Sharma
Rahul Sharma

Reputation: 5824

Alternatively you can create it directly using SparkSession-

x = [[1,10],[2,14],[3,17]]
df = spark.createDataFrame(data=x, schema = ["id","value"])
df.printSchema()
df.show()

Upvotes: 2

Poonam
Poonam

Reputation: 679

x = [[1,10],[2,14],[3,17]]
df = sc.parallelize(x).toDF(['ID','VALUE'])
df.show()

Upvotes: 7

Related Questions