CODEWITHSUNDEEP
javascriptcallbackcasperjs
Kunj Dab
Kunj Dab

Reputation: 5

Only the last iteration is executed when iterating through parametrized URLs with CasperJS

I have this piece of code:

var casper = require('casper').create();
var startID = 21;
var endID = 13533;
while (startID <= endID) {
    scrapPages(startID);
    startID++;
}

function scrapPages(i) {
    var str = "Starting to get the HTML for the problem" + i;
    console.log(str);

    var url = 'http://community.topcoder.com/stat?c=problem_statement&pm=' + i;
    casper.start(url, function() {
        this.echo(startID);
        var result = this.getHTML('td.problemText');
        casper.then(function() {
            var fileName = 'problem' + i + ".html";
            require('fs').write(fileName, result, 'w');
            this.echo("writing it to filename:  " + fileName)
        });
    });
}

casper.run();

All I'm doing is trying to parse Topcoder for all it's problems. I believe this question is not a duplicate of questions on Javascript Closures.

The problem is, the function scrapPages is running only once, the last iteration of while loop.

Can someone help me out why this is happening?

Upvotes: 0

Views: 33

Answers (1)

Artjom B.
Artjom B.

Reputation: 61952

You have many starts, but only one run. Both should be used exactly once. You can call start at the beginning before the loop without an argument and then change casper.start inside of scrapPages to casper.thenOpen.

You can verify that by going into the code. When start is called all previously scheduled steps are lost.

Upvotes: 1

Related Questions