CODEWITHSUNDEEP
pythonjsondecodeurllib
BarJacks
BarJacks

Reputation: 129

Urllib request throws a decode error when parsing from url

I'm trying to parse the json formatted data from this url: http://ws-old.parlament.ch/sessions?format=json. My browser copes nicely with the json data. But requests always throw the following error:

JSONDecodeError: Expecting value: line 3 column 1 (char 4)

I'm using Python 3.5. And this is my code:

import json
import urllib.request

connection = urllib.request.urlopen('http://ws-old.parlament.ch/affairs/20080062?format=json')

js = connection.read()

info = json.loads(js.decode("utf-8"))
print(info)

Upvotes: 0

Views: 595

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1124508

The site uses User-Agent filtering to only serve JS to known browsers. Luckily it is easily fooled, just set the User-Agent header to Mozilla:

request = urllib.request.Request(
    'http://ws-old.parlament.ch/affairs/20080062?format=json',
    headers={'User-Agent': 'Mozilla'})

connection = urllib.request.urlopen(request)
js = connection.read()

info = json.loads(js.decode("utf-8"))
print(info)

Upvotes: 2

Related Questions