PHP: decoding json feed

Question

How do I decode a whole feed (around 10k items) without memory limit of php? I found code which breaks the feed into parts: http://pastebin.com/m43c3384f

How do I modify the code to work with the feed below:

GAF_update_projects_vertical_callback({"projects":{"count":2,"items":[{"id":502817,"name":"SEO","url":"http:\/\/www.getafreelancer.com\/affiliates\/ref.php?u=boon86&l=http%3A%2F%2Fwww.getafreelancer.com%2Fprojects%2FSEO%2FSEO.502817.html","start_unixtime":1252386582,"start_date":"Tue, 08 Sep 2009 01:09:42 -0400","end_unixtime":1252818582,"end_date":"Sun, 13 Sep 2009 01:09:42 -0400","buyer":{"url":"http:\/\/www.getafreelancer.com\/affiliates\/ref.php?u=boon86&l=http%3A%2F%2Fwww.getafreelancer.com%2Fusers%2F1094422.html","id":1094422,"username":"eyalbiber"},"state":"A","short_descr":"Hi,  My name is Eyal.  I'm looking for seo expert to work with him on long term relationship.  I need...","options":{"featured":0,"nonpublic":0,"trial":0,"fulltime":0,"for_gold_members":0,"hidden_bids":0},"budget":{"min":30,"max":250},"jobs":["SEO"],"bid_stats":{"count":0,"avg":false}},{"id":503874,"name":"websubmissionarticles","url":"http:\/\/www.getafreelancer.com\/affiliates\/ref.php?u=boon86&l=http%3A%2F%2Fwww.getafreelancer.com%2Fprojects%2FCopywriting%2Fwebsubmissionarticles.html","start_unixtime":1252386528,"start_date":"Tue, 08 Sep 2009 01:08:48 -0400","end_unixtime":1252559328,"end_date":"Thu, 10 Sep 2009 01:08:48 -0400","buyer":{"url":"http:\/\/www.getafreelancer.com\/affiliates\/ref.php?u=boon86&l=http%3A%2F%2Fwww.getafreelancer.com%2Fusers%2F39628.html","id":39628,"username":"seexpert"},"state":"A","short_descr":"need some1 to rewrite articles by taking them from the net from various min 5 sources. he\/she can draft...","options":{"featured":0,"nonpublic":0,"trial":0,"fulltime":0,"for_gold_members":0,"hidden_bids":0},"budget":{"min":30,"max":250},"jobs":["Copywriting"],"bid_stats":{"count":0,"avg":false}}]}})

I tried this code below:

<?php
require("JSON.php");

$mochi_url = "http://api.getafreelancer.com/Project/Search.json?aff=480992&callback=GAF_update_projects_vertical_callback&order=submitdate&count=50&iads=true&nonpublic=0&charset=UTF-8";
$feed = file_get_contents($mochi_url);
 
$feed_start=strpos($feed,"[{");
 
$feed = substr($feed,$feed_start+1);
$feed = substr($feed,0,strlen($feed)-2);
$games = explode("{\"id\": \"",$feed);
unset($feed);
for($x=1;$x<=count($games);$x++){
     $games[$x] = "{\"name\": \"".$games[$x];
     $games[$x] = substr($games[$x],0,strlen($games[$x])-2);
     $games_decode = json_decode($games[$x], true);
 
     foreach($games_decode as $varname => $varvalue) {
          $mochi[$varname] = $varvalue;
     }
        
                $name = $mochi['name'];
 
 
                
                  echo "$x - Name: {$mochi['name']} <br>"; 
                unset ($games[$x]);
}
 
?> 

it returns

Warning: Invalid argument supplied for foreach() in /home/studiox/public_html/getafreelancer/auto.php on line 18 1 - Name:

any idea?

Answer 1

First of all, escape your json string. Like this:

$json_string = "{\"projects\":{\"count\":2,\"items\":[{\"id\":502817,\"name\":\"SEO\",\"url\":\"http:\/\/www.getafreelancer.com\/affiliates\/ref.php?u=boon86&l=http%3A%2F%2Fwww.getafreelancer.com%2Fprojects%2FSEO%2FSEO.502817.html\",\"start_unixtime\":1252386582,\"start_date\":\"Tue, 08 Sep 2009 01:09:42 -0400\",\"end_unixtime\":1252818582,\"end_date\":\"Sun, 13 Sep 2009 01:09:42 -0400\",\"buyer\":{\"url\":\"http:\/\/www.getafreelancer.com\/affiliates\/ref.php?u=boon86&l=http%3A%2F%2Fwww.getafreelancer.com%2Fusers%2F1094422.html\",\"id\":1094422,\"username\":\"eyalbiber\"},\"state\":\"A\",\"short_descr\":\"Hi,  My name is Eyal.  I'm looking for seo expert to work with him on long term relationship.  I need...\",\"options\":{\"featured\":0,\"nonpublic\":0,\"trial\":0,\"fulltime\":0,\"for_gold_members\":0,\"hidden_bids\":0},\"budget\":{\"min\":30,\"max\":250},\"jobs\":[\"SEO\"],\"bid_stats\":{\"count\":0,\"avg\":false}},{\"id\":503874,\"name\":\"websubmissionarticles\",\"url\":\"http:\/\/www.getafreelancer.com\/affiliates\/ref.php?u=boon86&l=http%3A%2F%2Fwww.getafreelancer.com%2Fprojects%2FCopywriting%2Fwebsubmissionarticles.html\",\"start_unixtime\":1252386528,\"start_date\":\"Tue, 08 Sep 2009 01:08:48 -0400\",\"end_unixtime\":1252559328,\"end_date\":\"Thu, 10 Sep 2009 01:08:48 -0400\",\"buyer\":{\"url\":\"http:\/\/www.getafreelancer.com\/affiliates\/ref.php?u=boon86&l=http%3A%2F%2Fwww.getafreelancer.com%2Fusers%2F39628.html\",\"id\":39628,\"username\":\"seexpert\"},\"state\":\"A\",\"short_descr\":\"need some1 to rewrite articles by taking them from the net from various min 5 sources. he\/she can draft...\",\"options\":{\"featured\":0,\"nonpublic\":0,\"trial\":0,\"fulltime\":0,\"for_gold_members\":0,\"hidden_bids\":0},\"budget\":{\"min\":30,\"max\":250},\"jobs\":[\"Copywriting\"],\"bid_stats\":{\"count\":0,\"avg\":false}}]}}";

PHP has native JSON support. You can decode JSON string directly in PHP associative array using json_decode():

$result = json_decode($json_string, true);

Testing:

echo $result['projects']['items'][0]['name'];
echo '<br>';
echo $result['projects']['items'][0]['url'];

Result:

SEO
http://www.getafreelancer.com/affiliates/ref.php?u=boon86&l=http%3A%2F%2Fwww.getafreelancer.com%2Fprojects%2FSEO%2FSEO.502817.html

UPDATE:

Answering question: What to do if i have many projects inside the JSON feed and i want to loop each one to store in mysql.

This is really not related to JSON, but rather to PHP arrays. There is a lot of handy function embedded into PHP that you may use. In this case all you need is to know array element count for your projects (read: amount of projects), then extract all the data out of your array using simple loop.

for($i = 0; $i < count($result['projects']['items']); $i++) {
  $project_name = $result['projects']['items'][$i]['name'];
  $project_id   = $result['projects']['items'][$i]['id'];
  $project_date = $result['projects']['items'][$i]['start_date'];

  // save / print / do whatever you want with this data
  echo "Project : $project_name [$project_id] - $project_date<br>";
}

P.S. Nice PHP array step-by-step tutorial is here.

Answer 2

If you're just running out of memory, try changing the memory_limit parameter in php.ini to something larger than the default (the default might be 8 MB, but I'm not certain).

The Increase PHP memory limit page on drupal.org has examples of other ways to increase the memory limit.