CODEWITHSUNDEEP
c#winformsvisual-studio-2010web-scraping
confusedMind
confusedMind

Reputation: 2653

How to get all urls from a webpage belonging to same Domain

Currently i am using this code to get the above :

                     Uri baseUri = new Uri(url);
                    Uri myUri = new Uri(baseUri, strRef);
                    domain = baseUri.Host;

                    Console.WriteLine(myUri.ToString());
                    strRef = myUri.ToString();
                    if (strRef.Contains(domain))
                    {
                     //THIS MEANS IT BELONGS TO SAME DOMAIN...
                    }

But using this code i am having some issue like suppose we have a main url = http://www.xxx.co.uk Then the above code also treats a url like http://www.news.xxx.co.uk as external link ? Is this correct should it do that if not any one know a better solution for this?

Upvotes: 0

Views: 413

Answers (1)

Sam
Sam

Reputation: 2917

I think you are in the correct path. But, to grab the latter mentioned URL (http://www.news.xxx.co.uk/) you could do a quick fix like this.

domain = baseUri.Host.Replace("www.", string.Empty);

Cheers!

vote if helpful.

Upvotes: 1

Related Questions