CODEWITHSUNDEEP
javascriptc#cefsharpchromium-embedded
Pankaj Agarwal
Pankaj Agarwal

Reputation: 137

Why is EvaluateScriptAsync returning an empty expando object in CefSharp

I am trying to write an application in C# using CefSharp. My intention is to fetch all the links on the given page eg, 

https://wixlabs---dropbox-folder.appspot.com/index?instance=lp5CbqBbK6JUFzCW2hXENEgT4Jn0Q-U1-lIAgEbjeio.eyJpbnN0YW5jZUlkIjoiYjNiNzk5YjktNjE5MS00ZDM0LTg3ZGQtYjY2MzI1NWEwMDNhIiwiYXBwRGVmSWQiOiIxNDkyNDg2NC01NmQ1LWI5NGItMDYwZi1jZDU3YmQxNmNjMjYiLCJzaWduRGF0ZSI6IjIwMTgtMDEtMjJUMTg6Mzk6MjkuNjAwWiIsInVpZCI6bnVsbCwidmVuZG9yUHJvZHVjdElkIjpudWxsLCJkZW1vTW9kZSI6ZmFsc2V9&target=_top&width=728&compId=comp-j6bjhny1&viewMode=viewer-seo

When I load the page and open the dev tools and execute

document.getElementsByTagName('a');

in the dev tools I get 374 results. Next I execute the following code from BrowserLoadingStateChanged:-

private async Task ProcessLinksAsync()
        {
            var frame = browser.GetMainFrame();
            var response = await frame.EvaluateScriptAsync("(function() { return document.getElementsByTagName('a'); })();", null);
            ExpandoObject result = response.Result as ExpandoObject;

            Console.WriteLine("Result:" + result);//What do I do here?
        }

I get an expando object which seems to contain nothing. I am saying this because I used a break point and inspected the object. I have gone through https://keyholesoftware.com/2019/02/11/create-your-own-web-bots-in-net-with-cefsharp/ , https://github.com/cefsharp/CefSharp/wiki/General-Usage#javascript-integration and the questions on SO but was unable to solve my problem. Am I doing something wrong here? My actual intention is to fetch the links and then navigate to them. Thanks in advance.

EDIT: I used the following script in browser and dev tools both return 187 results which is correct.

(function() { 
    var links=document.getElementsByClassName('file-link'); 
    var linksArray = new Array(); 
    for (var i = 0; i < links.length; i++) { 
        linksArray[i] = String(links[i].href); 
    } 
    return linksArray; 
})();

But in my application I get a 0 length array.

EDIT-2: I used the following code to get the DOM:-

public void OnContextCreated(IWebBrowser browserControl, IBrowser browser, IFrame frame)
        {
            ContextCreated?.Invoke(this, frame);
            const string script = "document.addEventListener('DOMContentLoaded', function(){ alert(document.links.length); });";

            frame.ExecuteJavaScriptAsync(script);
        }

For every other site I tried the code was successful except the URL mentioned above. Could any one possibly tell me what could be possibly wrong as the DOM is loaded in the dev tools and fully accessible. So, I guess something might be missing in my code. Thanks again.

Upvotes: 0

Views: 1525

Answers (1)

Reza Aghaei
Reza Aghaei

Reputation: 125237

You need to wait for the page loads. Also if the page loads data using ajax, you need to wait a bit to data also load. Then you need to shape the result to a custom javascript object.

ChromiumWebBrowser browser;
protected override void OnLoad(EventArgs e)
{
    base.OnLoad(e);
    browser = new ChromiumWebBrowser(
        "https://google.com/"); // Tried with your URL. 
    browser.LoadingStateChanged += Browser_LoadingStateChanged;
    browser.Dock = DockStyle.Fill;
    Controls.Add(browser);
}

private async void Browser_LoadingStateChanged(object sender,
    LoadingStateChangedEventArgs e)
{
    if (!e.IsLoading)
    {
        await Task.Delay(5000); //Just for pages which use ajax loading data
        var script = @"
        (function () {
            var data = document.getElementsByTagName('a');
            return Array.from(data, a => ({href:a.href, innerText:a.innerText}));
        })();";
        var result = await browser.EvaluateScriptAsync(script);
        var data = (IEnumerable<dynamic>)result.Result;

        MessageBox.Show(string.Join("\n", data.Select(x=>$"{x.href}").Distinct()));
    }
}

Upvotes: 2

Related Questions