CODEWITHSUNDEEP
pythonpython-requests
Halim stack
Halim stack

Reputation: 13

How to identify elements needed for requests in python?

I have a web url https://www.blibli.com/backend/content/promotions/blm-ranch-market where when I opened it on chrome returns a json.

However when I tried using requests in python and tried using headers including the cookie from headers it does not work. What is missing? How do we know what's needed for the request?I check in sources there's no payload only headers, I assumed by this code it will work. However using the same headers does not work.

import requests

resp = requests.get('https://www.blibli.com/backend/content/promotions/blm-ranch-market')
if resp == 200:
    print(resp.json())

Upvotes: 1

Views: 165

Answers (3)

Alexander
Alexander

Reputation: 17291

It worked fine for me after adding a User-Agent header.

import requests

headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
}

response = requests.get('https://www.blibli.com/backend/content/promotions/blm-ranch-market', headers=headers)

print(response.status_code)
print(response.json())

output:

200
{'code': 200,
 'data': {'components': [{'id': '8ab055d680424814018045bccc5e42ce',
                          'name': 'PAGE_ANCHOR_TABBING',
                          'parameters': [{'alt': 'Produk Terlaris',
                                          'id': '8ab055d680424814018045bccc5e42d2',
                                          'image': 'https://www.static-src.com/siva/asset//02_2022/Icon_TBL_musthave.png',
                                          'imageRatio': '1:1',
                                          'isMerchant': False,
                                          'name': 'image_4',
                                          'newTab': False,
                                          'requireLocation': False,
                                          'sequence': 0,
                                          'showProductTag': False,
                                          'sumOfProduct': 0,
                                          'title': 'Produk Terlaris',
                                          'type': 'IMAGE',
                                          'url': 'PROMO_1_BANNER-24',
                                          'use2HdProduct': False},
                                         {'alt': 'Produk Pilihan',
                                          'id': '8ab055d680424814018045bccc5e42d3',
                                          'image': 'https://www.static-src.com/siva/asset//02_2022/Icon_TBL_popular.png',
                                          'imageRatio': '1:1',
                                          'isMerchant': False,
                                          'name': 'image_5',
                                          'newTab': False,
                                          'requireLocation': False,
                                          'sequence': 0,
                                          'showProductTag': False,
                                          'sumOfProduct': 0,
                                          'title': 'Produk Pilihan',
                                          'type': 'IMAGE',
                                          'url': 'PROMO_1_BANNER-26',
                                          'use2HdProduct': False},
                                         {'alt': 'Vouchers',
                                          'id': '8ab055d680424814018045bccc5e42cf',
                                          'image': 'https://www.static-src.com/siva/asset//02_2022/BPD-FEB-ICON-CASHBACK.jpg',
                                          'imageRatio': '1:1',
                                          'isMerchant': False,
                                          'name': 'image_1',
                                          'newTab': False,
                                          'requireLocation': False,
                                          'sequence': 0,
                                          'showProductTag': False,
                                          'sumOfProduct': 0,
                                          'title': 'Vouchers',
                                          'type': 'IMAGE',
                                          'url': 'VOUCHERS_GEN_2-6',
                                          'use2HdProduct': False},
...

Upvotes: 1

relent95
relent95

Reputation: 4731

It's because that site does not support HTTP/1.1 and requests(and its backend http client, http.client in the standard library) does not support HTTP/2.

If you need a HTTP/2 client library, try other library, such as HTTPX.

Upvotes: 0

ankitgupta5294
ankitgupta5294

Reputation: 111

If you want to get the contents of the page, you might want to try resp.content.

Upvotes: 0

Related Questions