CODEWITHSUNDEEP
javascriptdomweb-scrapingphantomjs
DerManoMann
DerManoMann

Reputation: 529

copying data from one page to another using phantomJS

I am trying to copy some data from one processed web page into a new one that I want to export. The background is that I need to scrape parts of a page and need to build a new page with parts of the original page. The problem seems that phantomJs includeJs() and evaluate() methods are sandboxed and I can't see a proper way to import DOM from one page to another.

I have some test code that looks like this, with page being the original and out the new page:

    ....
    var title = page.evaluate(function() {
        return title = document.getElementById('fooo').innerHTML;
    });
    console.log('page title:' + title);
    //fs.write('c:/Temp/title.js', "var title = '" + title + "';", 'w');

    var out = new WebPage;
    out.viewportSize = page.viewportSize;
    out.content = '<html><head></head><body><div id="wrapper"></div><p>done</p></body></html>';
    out.includeJs('c:/Temp/title.js', function() {
        var p = document.createElement('p');
        p.appendChild(document.createTextNode(title));
        document.getElementById('wrapper').appendChild(p);
    });
    ...

Upvotes: 2

Views: 770

Answers (1)

nrabinowitz
nrabinowitz

Reputation: 55678

The function in your last includeJs call here won't work - as you note, it's sandboxed, and that means that closures won't work, so title won't be defined. A method of passing variables to page.evaluate is noted as a feature request, but isn't available as of PhantomJS v.1.4.1.

The general way I get around this is by using the Function constructor, which allows you to create a function using a string:

var myVar = {some:"values", I:"want to pass into my page"},
    test = new Function("window.myVar = " + JSON.stringify(myVar));
page.evaluate(test);

Now you can evaluate a function like the one you have, referencing myVar in the sandbox, and your data will be available in the client scope.

Upvotes: 3

Related Questions