CODEWITHSUNDEEP
pythonxmlhttprequestweb-scraping
user1956609
user1956609

Reputation: 2202

Save all XHR responses during session

I'm browsing through the Angie's List website with my Fiddler extension open in Chrome, and after each page load, Fiddler captures an XHR response for that page (I believe it's just a pixel tracker indicating the event that I visited a new page). I'd like to be able to capture the content of these responses automatically in a CSV file. So for example if I run "python getXHR.py http://www.angieslist.com" I'd want my csv output file to append:

angieslist.com,http://536371345.log.optimizely.com/event?a=536371345&d=536371345&y=fal...

How can I do this? I know Python better than other languages, but other languages are fine. If there is a way to do this directly through Fiddler/Firebug that's fine too.

Upvotes: 0

Views: 1692

Answers (1)

furas
furas

Reputation: 142939

With NetExport (extension for firebug) you can save all requests as JSON and then you could use Python (and module json) to find requests with X-Requested-With: XMLHttpRequest

BTW: NetExport has Auto Export

Upvotes: 1

Related Questions