CODEWITHSUNDEEP
arraysapache-sparkpysparkexpandjson-extract
ShimsyV
ShimsyV

Reputation: 1

Extracting data from array in json data using pyspark

I am new to pyspark and need some help with trying to extract data within an array. The data is from a form like this form imageThe schema is here schema I have managed to extract data till A1, A2, A3. But unable to extract B1, B2, B3. The path to B1 is Item.Items.L.M.Name.M.B.M.Items.L.M.B1.S

This is the code I am trying to use to extract the data

bitcode = data.select(col('item.guid.s').alias('wID'), col('item.ReportID.s').alias('ReportID'), col('Item.Items.L.M.Name.M.B.M.Items.L.M.B1.S').alias('B1')) bitcodes.show()

error I get is: No such struct field B1 in M

Upvotes: 0

Views: 116

Answers (0)

Related Questions