CODEWITHSUNDEEP
scaladataframeapache-sparkapache-spark-sql
ark
ark

Reputation: 35

Spark dataframe convert all the columns into json format and then modify json structure

Converted my dataframe into json using df.toJSON

After json conversion the schema looks like this :

root
 |-- value: string (nullable = true)

Whats the best way to make the current schema(i.e., all root level attributes; in this case the 'value' column) nested into a new root level json key ( called 'data' ) and add new attributes at root level. How to achieve this in scala.

Upvotes: 0

Views: 207

Answers (1)

s.polam
s.polam

Reputation: 10362

Please check below code.

scala> df.toJSON.select(struct($"value").as("data")).printSchema
root
 |-- data: struct (nullable = false)
 |    |-- value: string (nullable = true)

Upvotes: 1

Related Questions