CODEWITHSUNDEEP
javascriptcasperjs
tabibito
tabibito

Reputation: 13

How can I deal with asynchronous requests involving modal popups in Casperjs?

Trying to iterate through a list of links that open modal popups, I'm running into an issue with the asynchronous nature of Javascript. I can loop through the links, and I can get Casperjs to click on all of the links. The popup opens up well (and I need to save the content of that popup). However, my code leads to Casperjs skipping every few links -- I suspect that's because of the delay. I need to be sure that every link is clicked and every popup saved. Any hint is highly appreciated!

I'm aware of Casperjs wait and waitForSelector functions, but no matter where I put them -- it still skips some popups. I suppose the reason for this behaviour is the delay, but increasing/decreasing the wait values and places where I tell casperjs to wait don't help.

this.then(function(){
    x = 0;
        this.each(links,function(self,link){
        // I only need links that contain a certain string
        if(link.indexOf('jugyoKmkName')>=0) {
            var coursetitle = linktexts[x];
            this.clickLabel(linktexts[x], 'a');
               this.wait(2000, function() {
                var coursetitleSplit = coursetitle.split(' ');
                var courseid = coursetitleSplit[0];

                //this logs the title and id in a file. Works perfectly
                var line = courseid+'   '+coursetitle+' \\n';
                        fs.write('/myappdirectory/alldata.txt', line, 'a');

                //this logs the popup contents -- but it's completely out of sync
                var courseinfo = this.getElementInfo('.rx-dialog-large').html
                fs.write('/myappdirectory/'+courseid+'.html', courseinfo, 'w');
            }); 
        } 
        x++;
     }); 
});

I'm logging two things here -- the link text (and some more information) in a running log file. That's working fine -- it catches every link correctly. The link text contains a unique id, which I'm using as a file name to save the popup contents. That's only working on every nth popup -- and the popup contents and the id are out of sync.

To be precise: The first 10 ids in the list are:

20000 -- saved with this id, but contains data of popup 20215 20160 -- saved with this id, but contains data of popup 20307 20211 -- saved with this id, but contains data of popup 20312 20214 ...etc (saved, but with popup from an ID way further down the list) 20215 20225 20235 20236 20307 20308

Obviously, I need the file 2000.html to save the contents of the popup with the ID 20000, 20160 with the contents of 20160 etc.

Upvotes: 1

Views: 49

Answers (1)

Ryan Jenkins
Ryan Jenkins

Reputation: 890

Presumably this.each(links,...) will run the callback synchronously rather than waiting for each this.wait() call to complete. Instead you'll want to wait until you've written your data to the filesystem before processing the next link. Consider this code instead:

this.then(function() {
  function processNthLink(i) {
    var self = this;
    var link = links[i];

    if (link.indexOf('jugyoKmkName')>=0) {
      var coursetitle = linktexts[i];
      self.clickLabel(linktexts[i], 'a');
      self.wait(2000, function() {
        var coursetitleSplit = coursetitle.split(' ');
        var courseid = coursetitleSplit[0]; 
        var line = courseid+'   '+coursetitle+' \\n';
        fs.write('/myappdirectory/alldata.txt', line, 'a'); 
        var courseinfo = self.getElementInfo('.rx-dialog-large').html
        fs.write('/myappdirectory/'+courseid+'.html', courseinfo, 'w');

        if (i < links.length) {
          processNthLink(i+1);
        }
      }); 
    } else if (i < links.length) {
      processNthLink(i+1);
    }
  }

  processNthLink(0);
});

In this case the the next link will only be processed after the timeout and write to FS has been completed. In the case that the link doesn't contain the expected string, the next link is processed immediately.

Upvotes: 1

Related Questions