CODEWITHSUNDEEP
c#html-agility-pack
Mohammad
Mohammad

Reputation: 1

i have error with web scraping and html agility pack in c#

 private void button1_Click(object sender, EventArgs e)
    {
        var html = @"https://html-agility-pack.net/from-web";
        HtmlWeb web = new HtmlWeb();
        var htmldoc = web.Load(html);
        var nod = htmldoc.DocumentNode.SelectSingleNode("//head/title");
        textBox1.Text = "Node Name: " + nod.Name + "\n" + nod.OuterHtml;
    }

at first i want learning webscraoing with c# and html agility pack but when I run my first code this meassage will appear:

{An unhandled exception of type 'System.Net.WebException' occurred in HtmlAgilityPack.dll

Additional information: The underlying connection was closed: An unexpected error occurred on a send.}

Upvotes: 0

Views: 824

Answers (1)

user8126467
user8126467

Reputation:

The underlying connection was closed: An unexpected error occurred on a send

This problem occurs when the client computer cannot send an HTTP request because the connection has been closed or unavailable.

Since you're trying to accessing HTTPS page you may had to set security protocol before creating request.

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
var html = @"https://html-agility-pack.net/from-web";
HtmlWeb web = new HtmlWeb();

This will prefer TLS 1.2 but still allow 1.1 and 1.0 (to prevent failure because some sites doesn't offer TLS 1.2).

Upvotes: 1

Related Questions