CODEWITHSUNDEEP
pythonjson
Araf
Araf

Reputation: 273

How to extract specific data from JSON object using python?

I'm trying to scrape a website and get items list from it using python. I parsed the html using BeaufitulSoup and made a JSON file using json.loads(data). The JSON object looks like this:

{ ".1768j8gv7e8__0":{ 
    "context":{ 
       //some info
    },
    "pathname":"abc",
    "showPhoneLoginDialog":false,
    "showLoginDialog":false,
    "showForgotPasswordDialog":false,
    "isMobileMenuExpanded":false,
    "showFbLoginEmailDialog":false,
    "showRequestProductDialog":false,
    "isContinueWithSite":true,
    "hideCoreHeader":false,
    "hideVerticalMenu":false,
    "sequenceSeed":"web-157215950176521",
    "theme":"default",
    "offerCount":null
 },
 ".1768j8gv7e8.6.2.0.0__6":{ 
    "categories":[ 

    ],
    "products":{ 
       "count":12,
       "items":[ 
          { 
             //item info
          },
          { 
            //item info
          },
          { 
            //item info
          }
       ],
       "pageSize":50,
       "nextSkip":100,
       "hasMore":false
    },
    "featuredProductsForCategory":{ 

    },
    "currentCategory":null,
    "currentManufacturer":null,
    "type":"Search",
    "showProductDetail":false,
    "updating":false,
    "notFound":false
 }
}

I need the items list from product section. How can I extract that?

Upvotes: 0

Views: 11561

Answers (3)

Max Voisard
Max Voisard

Reputation: 1942

I'll just call the URL of the JSON file you got from BeautifulSoup "response" and then put in a sample key in the items array, like itemId:

import json
json_obj = json.load(response)
array = []
for i in json_obj['items']:
   array[i] = i['itemId']
print(array)

Upvotes: 1

Jacek Rojek
Jacek Rojek

Reputation: 1122

import json packagee and map every entry to a list of items if it has any:

This solution is more universal, it will check all items in your json and find all the items without hardcoding the index of an element

import json

data = '{"p1": { "pathname":"abc" },  "p2": { "pathname":"abcd", "products": { "items" : [1,2,3]} }}'

# use json package to convert json string to dictionary
jsonData = json.loads(data)
type(jsonData) # dictionary

# use "list comprehension" to iterate over all the items in json file
# itemData['products']["items"] - select items from data
# if "products" in itemData.keys() - check if given item has products 
[itemData['products']["items"] for itemId, itemData in jsonData.items() if "products" in itemData.keys()]

Edit: added comments to code

Upvotes: 1

ByteFlowr
ByteFlowr

Reputation: 500

Just do:

products = jsonObject[list(jsonObject.keys())[1]]["products"]["items"]

Upvotes: 1

Related Questions