CODEWITHSUNDEEP
asp.net
stighy
stighy

Reputation: 7170

How to download a html page via asp.net

how can i download a web page from my web app, then reading "title" and "description" metatag ? Like a web crawler, but in Asp.net and called by ..an asp.net web page ?

Thanks!

Upvotes: 1

Views: 2073

Answers (2)

Scott Mitchell
Scott Mitchell

Reputation: 8759

You can do a screen scrape of an external URL in .NET using the WebClient class, which you'll find in the System.Net namespace. Use the DownloadData method to download the content from a specified URL. The downloaded data comes down as a byte array, but you can convert this to a string.

The following snippet shows how to use WebClient to grab the HTML from my blog's homepage, http://scottonwriting.net/sowblog/default.aspx:

// Create a new WebClient instance.
WebClient myWebClient = new WebClient();

// Download the markup from 
byte[] myDataBuffer = myWebClient.DownloadData("http://scottonwriting.net/sowblog/default.aspx");

// Convert the downloaded data into a string
string markup = Encoding.ASCII.GetString(myDataBuffer);

Once you have the markup you can use regular expressions or string searching methods to pick out the markup of interest.

Upvotes: 2

SLaks
SLaks

Reputation: 887451

Use the HTML Agility Pack and its HTMLWeb class.

Upvotes: 0

Related Questions