Reputation: 89
using (var wc = new WebClient())
{
Uri urls = new Uri(url);
wc.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36";
dokuman.Load(wc.OpenRead(urls), Encoding.UTF8);
}
dokuman.Load(wc.OpenRead(urls), Encoding.UTF8);
The remote server returned an error: (503) Server Unavailable.
Sometimes I get this error. How can we solve this?
Upvotes: 4
Views: 34735
Reputation: 3865
I use this utility function it supports proxies too
public static string GetPageHtml(string link, System.Net.WebProxy proxy = null)
{
System.Net.WebClient client = new System.Net.WebClient() { Encoding = Encoding.UTF8 };
client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");
if (proxy != null)
{
client.Proxy = proxy;
}
using (client)
{
try
{
return client.DownloadString(link);
}
catch (Exception ex)
{
return null;
}
}
}
Try it, if it doesn't work then you probably got blocked by google because you were making requests fast, to avoid getting blocked you should use web proxies or make reasonable pauses between your requests.
Example
var Html = GetPageHtml("https://www.google.com.tr/search?sclient=psy-ab&site=&source=hp&q=cars+&btnG=Ara");
Upvotes: 2