CODEWITHSUNDEEP
phpjsonyql
Phrosty30
Phrosty30

Reputation: 127

Using yql and php to scrape content

been fooling around with YQL trying to understand it a little better. I've managed to pull the info I want from an external site and get a results node in the YQL console but have thus been unsuccessful in displaying the results on my local development server.

What I ultimately want to do is try and put this into a wordpress function so I can call it on a page (for example the standings page).

The code I used in php (edit:: I changed the code to this)

ini_set('display_errors', 1);
    ini_set('log_errors', 1);
    error_reporting(E_ALL);

    $yql_base_url ="http://query.yahooapis.com/v1/public/yql";
    $yql_query = 'SELECT * FROM html WHERE url="http://www.nwjhl.com/leagues/standingsTotals.cfm?leagueID=15654&clientID=4594" AND xpath=\'//table[@class="yuiboxscores"]/tbody/tr\'';

    $yql_query_url = $yql_base_url . "?q=" . urlencode($yql_query);
    $yql_query_url .= "&format=json";

    // set up the cURL
    $c = curl_init();
    curl_setopt($c, CURLOPT_URL, $yql_query_url);
    curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($c, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($c, CURLOPT_SSL_VERIFYHOST, false);

    // execute the cURL
    $rawdata = curl_exec($c);
    curl_close($c);

    // Convert the returned JSON to a PHP object
    $data = json_decode($rawdata);

I've managed to get the data Im looking for now Im just having trouble pulling the data in as Im assuming its because the "tr" is in an array.

Ive been probably staring at its probably a little to complex for figuring with YQL for me, I figure a for loop is needed, I just unsure of how to reference the [content] and then of course all the other numbers. I tried to debug with

$results = $data->query->results->td;
    echo '<pre>' . print_r($results) . '</pre>';

Im not sure how I would be able to loop through all these objects a arrays to display [content] (which is the team) and then of course there statistics. Continuing to work on this hopefully I can figure out

Upvotes: 3

Views: 1130

Answers (1)

Andrew Edvalson
Andrew Edvalson

Reputation: 7878

The json you are getting back from that URL is not what you expect it to be. Try 

echo $output

right after curl_close($ch);

then running it through a json formatter (google it). $data->query->results works, but there is no child called Result, it just seems to be a jsonified html table of some sort.

Once you see the JSON in a more human readable format your problem might make more sense to you.

Per your Edit - Try this after you json_decode the raw data. I'm not sure exactly what pieces of data you are after, but I'm guessing it will be in the objects you are dumping with var_dump

$topics = $data->query->results->table->tbody->tr;
foreach ($topics as $topic)
   {
   echo "row:";
   var_dump($topic);
   }

Here's a more specific example of how to loop through this and find

elements

$topics = $data->query->results->table->tbody->tr;
foreach ($topics as $topic)
   {
   $data = $topic->td;
   foreach ($data as $element)
      {
      if (array_key_exists('a', $element))
         {
         echo "Team Name: " . $element->a->content . "\n";
         }
      if (array_key_exists('p', $element))
         {
         echo "Found P: " . $element->p . "\n";
         }
      }
   }

Upvotes: 3

Related Questions