How to extract "response" data from a specific XHR file in networks traffic tab of chrome using python

screenshot

I am working on a project of data scrapping, and somehow the data I want from this dynamic website is available in the XHR file named query-list which the browser receives when I click on the next page button in the page every time, and I am currently using selenium to open webpage and click on the buttons before scrapping. It would be really helpful if I could access the response data of the latest XHR file formed under network tab in inspect (developer tools) and extract all text. I don't want use any 3rd party API service. If chrome can show the response data in the XHR file, maybe selenium or something can fetch it for me.

I have tried to selenium wire also, but don't know how to get the exact thing I want. I just want to get the data under response section of the XHR file formed in the networks tab in inspect every time my selenium clicks on the "next page" button

Upvotes: 0

Views: 113

Answers (0)

Related Questions