CODEWITHSUNDEEP
asp.netpython-3.xweb-scraping
Soham Parekh
Soham Parekh

Reputation: 23

Scraping data from .aspx page

I want to scrape data from a .aspx page, and I wanted to know if there was any way to do this apart from using selenium to select all possible options and click the submit button. I tried looking at the network section but did not understand where the data is coming from. Could anyone help me out?

Here is a link to the page:
https://www.lgindiasocial.com/microsites/brand-store-web-five/locate.aspx

Upvotes: 1

Views: 727

Answers (1)

codeandcloud
codeandcloud

Reputation: 55200

There is no other way than simulating option changes and click event.

Reason:

The page is rendered with ASP.NET Ajax (WebForms). This was Microsoft implementation of Ajax many, many years back. Most people(if not all) consider Webforms obslete. The section under your consideration uses, UpdatePanel

Enables sections of a page to be partially rendered without a postback.

It does not return an xml or json as one normally expect from an AJAX request. Instead, it returns the new rendered html and place it inside the UpdatePanel.

The request will be application/x-www-form-urlencoded and the response will be text/plain. It also send the `ViewState (What Is View State And How It Works In ASP.NET).

If you view source, you will understand that the UpdatePanel returns new html for the content inside <div id="UpdatePanel1"> for each xhr HTTP POST request in the network tab.

Upvotes: 2

Related Questions