CODEWITHSUNDEEP
pythonscrapy
Darshan Chaudhary
Darshan Chaudhary

Reputation: 2233

Spider for parsing JSON in scrapy

I have been using scrapy for some time now and I wish to know what is the best way to parse JSON. There are specially designed spiders for XML and CSV in scrapy viz. XMLFeedSpider and CSVFeedSpider. Is there any such spider for JSON too ?

Upvotes: 1

Views: 571

Answers (1)

GHajba
GHajba

Reputation: 3691

There is no default JSON parser for Scrapy, but you can use the json module to parse your response and convert it to items:

jsonresponse = json.loads(response.body_as_unicode())

However if you plan to write a general solution like CSVFeedSpider feel free to contribute it to the Scrapy project but I guess json.loads does the trick of consuming JSON responses.

Upvotes: 1

Related Questions