CODEWITHSUNDEEP
htmlweb-scrapingyqlweb-content
user7923319
user7923319

Reputation:

Getting null result every time when trying to scrape web content using YQL

I'm using YQL to fetch web content from an external website. But, despite of using proper xpath value and format as JSON too, I'm always getting result as null. I'm trying to get content for this below website. If YQL has any problems, can anyone suggest some alternative to YQL? I have tried this so far. Please have a look at it.

var site = "http://www.amazon.in/Seiko-Premier-Analog-Blue-Watch/dp/B012T413GO?_encoding=UTF8&ref_=cm_sw_r_wa_apa_i_5c5uzbQG5A293";
site = site.replace('m.', '')
var yql = "SELECT * FROM html WHERE url='" + site + "' AND xpath='//title|//head/meta'";
var resturl = "http://query.yahooapis.com/v1/public/yql?q=" + encodeURIComponent(yql) + "&format=json";

$.getJSON(resturl,function(data){
    console.log(data);
})

http://www.amazon.in/Seiko-Premier-Analog-Blue-Watch/dp/B012T413GO?_encoding=UTF8&ref_=cm_sw_r_wa_apa_i_5c5uzbQG5A293

Upvotes: 0

Views: 297

Answers (1)

Mauricio Arias Olave
Mauricio Arias Olave

Reputation: 2575

Here is the complete example, but first you need to:

  • Use https (http will likely return null or error).
  • Remember that you're getting meta tags, so if you need or try to show the results in HTML, you'll see nothing, hence I use the console.

var site = "https://www.amazon.in/Seiko-Premier-Analog-Blue-Watch/dp/B012T413GO?encoding=UTF8&ref=cm_sw_r_wa_apa_i_5c5uzbQG5A293";
site = site.replace('m.', '');

var yql = "select * from htmlstring where url='" + site + "' AND xpath='//title|//head/meta'";

var resturl = "https://query.yahooapis.com/v1/public/yql?q=" + encodeURIComponent(yql) + "&format=json&diagnostics=true&env=store://datatables.org/alltableswithkeys&callback=";

$.getJSON(resturl, function(data) {
  console.log(data.query.results.result);
})
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script>
<div id="divContent"><i>Look the console - the results are not visible (they're only meta tags):</i></div>

Upvotes: 0

Related Questions