CODEWITHSUNDEEP
javascriptphantomjs
aaronm2112
aaronm2112

Reputation: 27

PhantomJS not returning any results

I'm using PhantomJS to scrape data from a webpage. PhantomJS is not returning anything from the evaluate method. The script just runs for a few seconds and then exits.

I've already checked to see if PhantomJS is connecting to the page -- it is. PhantomJS is also able to grab the page title. I've already double-checked the class I'm looking for, yes -- I'm spelling it correctly.

var page = require('webpage').create();

page.open('http://www.maccosmetics.com/product/13854/36182/Products/Makeup/Lips/Lipstick/Giambattista-Valli-Lipstick', function(status) {
    page.includeJs("http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js", function() {
        waitFor(function()  {
            return page.evaluate(function() {
                $('.product__price').is(':visible');
            });
        }, function(){
            search = page.evaluate(function() {
                return  $('.product__price').text();
            });
            console.log(search)
        });
    });
    phantom.exit();
});

I don't know what's going wrong here.

Upvotes: 0

Views: 205

Answers (1)

Artjom B.
Artjom B.

Reputation: 61892

It's not showing you anything, because you're exiting too early. All functions (except evaluate()) that take a callback are asynchronous.

You're requesting to include jQuery in the page by calling page.includeJs(), you immediately exit PhantomJS. You need to exit when you're finished:

var page = require('webpage').create();

page.open('http://www.maccosmetics.com/product/13854/36182/Products/Makeup/Lips/Lipstick/Giambattista-Valli-Lipstick', function(status) {
    page.includeJs("http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js", function() {
        waitFor(function()  {
            return page.evaluate(function() {
                $('.product__price').is(':visible');
            });
        }, function(){
            search = page.evaluate(function() {
                return  $('.product__price').text();
            });
            console.log(search);
            phantom.exit();
        });
    });
});

Upvotes: 1

Related Questions