CODEWITHSUNDEEP
pythonweb-scraping
APPGIS
APPGIS

Reputation: 353

Python scraping response data

i would like to take the response data about a specific website. I have this site: https://enjoy.eni.com/it/milano/map/ and if i open the browser debuger console i can see a posr request that give a json response:

response image

how in python i can take this response by scraping the website? Thanks

Upvotes: 3

Views: 894

Answers (1)

Yohanes Gultom
Yohanes Gultom

Reputation: 3842

Apparently the webservice has a PHPSESSID validation so we need to get it first using proper user agent:

import requests
import json

headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36'
}
r = requests.get('https://enjoy.eni.com/it/milano/map/', headers=headers)
session_id = r.cookies['PHPSESSID']
headers['Cookie'] = 'PHPSESSID={};'.format(session_id)
res = requests.post('https://enjoy.eni.com/ajax/retrieve_vehicles', headers=headers, allow_redirects=False)
json_obj = json.loads(res.content)

Upvotes: 4

Related Questions