CODEWITHSUNDEEP
pythonweb-scrapingscrapy
quasarseeker
quasarseeker

Reputation: 167

Scraping and Handling 'application/amazonui-streaming-json' responses

Noticed that Amazon uses 'application/amazonui-streaming-json' as a content type to some of their responses.

I've been trying to scrape some price information from their store, but it seems Scrapy can't seem to recognize the response in text format.

When I try:

price_request = Request(url, self.parse_price, 
                       headers= {'Content-Type':'application/json; charset=utf-8'})

I get the following error:

AttributeError: 'Response' object has no attribute 'text'

Is there some way I change the response type in a format I can parse through with Scrapy?

Upvotes: 0

Views: 556

Answers (1)

quasarseeker
quasarseeker

Reputation: 167

The problem was I was trying to parse the request using response.text. Using response.body did the trick.

Upvotes: 1

Related Questions