CODEWITHSUNDEEP
pythonpython-3.xurllib2
washedev
washedev

Reputation: 107

How can I use url lib using a json file

I'm trying to get data from a json link, but I'm getting this error: TypeError: can't concat str to bytes

This is my code: 

l = "https://www.off---white.com/en/IT/men/products/omch016f18d471431088s"
url = (l+".json"+"?porcoiddio")
req = urllib.request.Request(url, headers)
response = urllib.request.urlopen(req)

size_opts = json.loads(response.decode('utf-8'))['available_sizes']

How can I solve this error?

Upvotes: 2

Views: 275

Answers (3)

KC.
KC.

Reputation: 3107

Your question answer is change your code to:

size_opts = json.loads(response.read().decode('utf-8'))['available_sizes']

Change at 2018-10-02 22:55 : I view your source code and found Response 503 , the reason why you got 503 is that request did not contain cookies:

req = urllib.request.Request(url, headers=headers)

you have update your headers.

headers.update({"Cookie":cookie_value})
req = urllib.request.Request(url, headers=headers) # !!!! you need a headers include cookies !!!!

Upvotes: 1

supertux
supertux

Reputation: 2189

You could have a go using requests instead?

import requests, json
l = "https://www.off---white.com/en/IT/men/products/omch016f18d471431088s"
url = (l+".json"+"?porcoiddio")
session = requests.Session()
session.mount('http://', requests.adapters.HTTPAdapter(max_retries=10))
size_opts = session.get(url, headers= {'Referer': 'off---white.com/it/IT/login', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'}).json()['available_sizes']

To check the response:

size_opts = session.get(url, headers= {'Referer': 'off---white.com/it/IT/login', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'})
print(size_opts)

Gives

<Response [503]>

This response means: "503 Service Unavailable. The server is currently unable to handle the request due to a temporary overload or scheduled maintenance"

I would suggest the problem isn't the code but the server?

Upvotes: 0

deif
deif

Reputation: 1389

you are providing the data argument by mistake …

you'll have to use a keyword argument for headers as otherwise the second argument will be filled with positional input, which happens to be data, try this:

req = urllib.request.Request(url, headers=headers)

See https://docs.python.org/3/library/urllib.request.html#urllib.request.Request for a documentation of Requests signature.

Upvotes: 0

Related Questions