CODEWITHSUNDEEP
c#webweb-scrapingtagshtml-agility-pack
SimplyBoost
SimplyBoost

Reputation: 15

SelectNodes in HtmlAgilityPack always return null

I'm trying to get data from this site: https://www.hltv.org/results, and it doesnt work. SelectNodes always returns null, I've tried using xpath, full xpath and so on and different things other than HtmlAgilityPack. Maybe its not the HtmlAgilityPack maybe the problem is in the tags. Please take a look at the code and view also the tags to see if if i copied it right: 

    HtmlWeb web = new HtmlWeb();
    var doc = web.Load("https://www.hltv.org/results");
    var teams = doc.DocumentNode.SelectNodes("//*[@class='team team-won']");

Please Help, Thanks!

Upvotes: 0

Views: 971

Answers (2)

Mesut Yildirim
Mesut Yildirim

Reputation: 11

Use load for only files. In this example, you ar loading an url, So you should use web.LoahHtml İf you do this. It will work

Upvotes: 1

ggeorge
ggeorge

Reputation: 1630

HtmlAgilityPack does not handle javascript, so you can use selenium to store the html source code via webDriver.

IWebDriver webDriver = new ChromeDriver();

webDriver.Url = "https://www.hltv.org/results";
var pageSource = webDriver.PageSource;

var doc = new HtmlDocument();
doc.LoadHtml(pageSource);

 var xPath = @"//*[@class='team team-won']";

 var node = doc.DocumentNode.SelectSingleNode(xPath);

 Console.WriteLine(node.InnerText);

Make sure that you have already downloaded and stored in same directory the webDriver.(i used chrome driver downloaded from here https://chromedriver.chromium.org/downloads)

Upvotes: 0

Related Questions