CODEWITHSUNDEEP
phpmysqljson
RawAnimal
RawAnimal

Reputation: 43

Parse JSON To Create SQL Insert Statements in PHP

I'm a newbie programmer trying to find my way in the world. I've got my hands on JSON data that I'm trying to parse out into SQL statements to populate multiple database tables. I would like to loop through each dimension of the array and pull out specific parts of it to create an INSERT statement I can just pass to MySQL. I'm not sure if this is the best way to populate separate tables with data from one JSON file but it's the only thing I can think of. MySQL tables are separated so there is a table for a person, a table for address type, a table for address, a table for phone, a table for email etc. This is to account for a person record having numerous phone numbers, email addresses etc.

I have been able to decode the JSON from an external URL. Here is the code and a sample of the output using print_r.

$json_string = 'http://....';

$jsondata = file_get_contents($json_string);

$data = json_decode($jsondata, TRUE);

1 Record Sample:

Array ( [objects] => Array ( [0] => Array ( [first_name] => Anthony [last_name] => Perruzza [name] => Anthony Perruzza [elected_office] => City councillor [url] => http://www.toronto.ca/councillors/perruzza1.htm [gender] => [extra] => Array ( ) [related] => Array ( [boundary_url] => /boundaries/toronto-wards/york-west-8/ [representative_set_url] => /representative-sets/toronto-city-council/ ) [source_url] => http://www.toronto.ca/councillors/perruzza1.htm [offices] => Array ( [0] => Array ( [tel] => 416-338-5335 ) ) [representative_set_name] => Toronto City Council [party_name] => [district_name] => York West (8) [email] => [email protected] [personal_url] => [photo_url] => ) ) [meta] => Array ( [next] => /representatives/?limit=1&offset=1 [total_count] => 1059 [previous] => [limit] => 1 [offset] => 0 ) )

JSON Code Sample:

{"objects": [
    {"first_name": "Keith",
     "last_name": "Ashfield",
     "name": "Keith Ashfield",
     "elected_office": "MP",
     "url": "http://www.parl.gc.ca/MembersOfParliament/ProfileMP.aspx?Key=170143&Language=E",
     "gender": "",
     "extra": {},
     "related": {
          "boundary_url": "/boundaries/federal-electoral-districts/13003/", 
          "representative_set_url": "/representative-sets/house-of-commons/"
     },
     "source_url": "http://www.parl.gc.ca/MembersOfParliament/MainMPsCompleteList.aspx?TimePeriod=Current&Language=E",
     "offices": [
         { "type": "legislature",
           "fax": "613-996-9955",
           "postal": "House of Commons\nOttawa, Ontario\nK1A 0A6",
           "tel": "613-992-1067"
         },
         { "type": "constituency",
           "fax": "506-452-4076",
           "postal": "23 Alison Blvd (Main Office)\nFredericton, New Brunswick\nE3C 2N5",
           "tel": "506-452-4110"
         }
     ],
     "representative_set_name": "House of Commons",
     "party_name": "Conservative",
     "district_name": "Fredericton",
     "email": "[email protected]",
     "personal_url": "",
     "photo_url": "http://www.parl.gc.ca/MembersOfParliament/Images/OfficialMPPhotos/41/AshfieldKeith_CPC.jpg"
    }
 ],
  "meta": {
       "next": "/representatives/house-of-commons/?limit=1&offset=1",
       "total_count": 307,
       "previous": null,
       "limit": 1,
       "offset": 0
   }
 }

Any help you can offer would be greatly appreciated. I've been pulling my hair out for the last few days trying to figure it out.

I've tried customizing code like the following to make it work but I haven't been able to hit the sweet spot. Please not, this code doesn't reference my data or variables. I deleted what didn't work for me. I'm just including it to give you an idea what I've tried.

foreach ($data as $item) {
    echo $item->{'first_name'} . "<br/>";
    echo $item->{'last_name'};
}

If you could point me in the direction of being able to parse out data from any level of the array it would be greatly appreciated.

Best,

S

Upvotes: 1

Views: 19474

Answers (3)

Juan Carlos Constantine
Juan Carlos Constantine

Reputation: 124

function JSONTOInsertSQL($table,$obj){
    $keys   = implode('`,`', array_map('addslashes', array_keys($obj)));
    $values = implode("','", array_map('addslashes', array_values($obj)));
    return "INSERT INTO `$table` (`$keys`) VALUES ('$values')";
}

Upvotes: 1

Martin Jstone
Martin Jstone

Reputation: 31

Here is something that you can use to get the structure of a json response. It works recursively so that it will create an entry to for each object as well as an entry in a separate table for each property of each object. I hope to get others feedback/enhancement on this as well to turn it into create sql statements.

    class DataDriller {

  var $ext_obj_to_parse;
  var $ext_type_name;
  var $data;
  var $recurse;
  var $ext_type_id;
  var $ext_related_id;
  var $type_id;
  var $auto_create;
  var $sql;
  var $error;
  var $controller;
  var $ExtType;
  var $ExtStructure;
  var $link;
  var $ext_source_id;

  function init($ExtType, $ExtStructure) {
    $this->ExtType = $ExtType;
    $this->ExtStructure = $ExtStructure;
  }

  function setup($ext_obj_to_parse, $ext_type_name, $ext_type_id = false, $ext_related_id = false, $auto_create = true, $ext_source_id) {
    $this->ext_obj_to_parse = $ext_obj_to_parse;
    $this->ext_type_name = $ext_type_name;
    $this->ext_type_id = $ext_type_id;
    $this->auto_create = $auto_create;
    $this->error = false;
    $this->ext_related_id = $ext_related_id;
    $this->ext_source_id = $ext_source_id;    



    if ($this->get_ext_type_data() === false) {
      if ($this->type_handling() === false) {
        $this->error_data();
      }
    }

    if (gettype($this->ext_obj_to_parse) == "object" || gettype($this->ext_obj_to_parse) == "array") {
      $this->to_struct();
    } else {
      //single variable and data
      $this->data[$this->ext_type_name] = gettype($this->ext_obj_to_parse);
      $this->sql = "replace into ext_structures (name, data_type, ext_type_id) values ('$this->ext_type_name', '" . gettype($this->ext_obj_to_parse) . "', " . $this->ext_type_id . ")";
      $this->sql_it();
    }
  }

  function get_ext_type_data() {
    if (is_numeric($this->ext_type_id)) {
      return true;
    } else if (strlen($this->ext_type_name) > 0) {
      $this->sql = "select id From ext_types where name = '" . $this->ext_type_name . "' limit 1";
      $this->ext_type_id = $this->sql_it('id');
      return $this->ext_type_id;
    } else {
      return false;
    }
  }

  function type_handling() {
    if ($this->auto_create == true && gettype($this->ext_type_name) === "string") {
      //$this->sql = "replace into types (name) values ('$this->ext_type_name')";
      //
      //$this->type_id = $this->sql_it();
      //if ($this->type_id !== 0) {
      //if ($this->ext_related_id) {
        $this->sql = "insert into ext_types (name, ext_source_id, parent_id) values ( '$this->ext_type_name',  $this->ext_source_id, '$this->ext_related_id')";
        $this->ext_type_id = $this->sql_it();

        $this->sql = "replace into ext_type_rel (ext_type_id_1, ext_type_id_2) values ($this->ext_type_id, $this->ext_related_id)";
        $this->sql_it();
      /*} else {
        $this->error = "Unable to obtain typeid from insert";
        $this->error_data();
        return false;
      }*/
    }
    //}
  }

  function to_struct() {
    //keys are not objects but values can be
    //always display keys, when value object - increase spacer - call self - reiterate
    // if value is not object complete 
    foreach ($this->ext_obj_to_parse as $key => $value) {
      if (gettype($value) == "object" || gettype($value) == "array") {
        //check to see if object exists within the database with the data definitions and methods
        //there are no existing data structure insert 
        //recurse into the drill-down again if it does not exist
        if (is_numeric($key) || $key == "data" || $key == "statuses") {
          $this->recurse = new DataDriller();
          if (!$this->ext_related_id > 0){ $this->ext_related_it = $this->ext_type_id; }

          $this->recurse->setup($value, $this->ext_type_name, $this->ext_type_id, $this->ext_related_id, true, $this->ext_source_id);
        } else {
          $this->recurse = new DataDriller();
          $this->recurse->setup($value, $key, false, $this->ext_type_id, true, $this->ext_source_id);
        }

        $this->data[$key] = $this->recurse->data;

        unset($this->recurse);
        //this is where we insert the relationship between objects here
      } else {
        //not an ojbect just a field of the existing object 
        $this->data[$key] = gettype($value);

        $this->sql = "replace into ext_structures (name, data_type, ext_type_id) values ('$key', '" . gettype($value) . "', " . $this->ext_type_id . ")";
        $this->sql_it();
      }
    }
  }

  function sql_it($field_name = false) {

    $VARDB_server = '192.168.10....';
    $VARDB_port = '3306';
    $VARDB_user = 'user';
    $VARDB_pass = 'pass';
    $VARDB_database = 'db_name';

    $this->link = mysql_connect("$VARDB_server:$VARDB_port", "$VARDB_user", "$VARDB_pass");
    if (!$this->link) {
      echo 'MySQL connect ERROR: ' . mysql_error();
      die();
    }
    $res = mysql_select_db("$VARDB_database");
    if (!$res) {
      echo mysql_error();
    }


    $res = mysql_query($this->sql, $this->link);
    if (mysql_error()) {
      $this->error = mysql_error() . " MYSQL reported an error " . $this->sql;
      CakeLog::write('datadriller', $this->sql . " error? " . mysql_error());
      die();
    }

    if ($field_name === false) {
      if (strpos($this->sql, 'insert') !== false || strpos($this->sql, 'replace') !== false) {
        $id = mysql_insert_id();
        return $id;
      } else {
        $this->error = "field name is requeired for getting results";
        $this->error_data();
        return false;
      }
    } else {
      if (mysql_num_rows($res) > 0) {
        $r = mysql_fetch_array($res);
        mysql_free_result($res);
        if (array_key_exists($field_name, $r)) {
          return $r[$field_name];
        } else {
          $this->error = "field name does not exist in result set";
          $this->error_data();
          return false;
        }
      } else {
        $this->error = "select statement returned no data ";
        return false;
      }
    }
  }

  function error_data() {
    echo "<B> $this->error MySQL error? <font color=red>" . mysql_error() . " </font> SQL: $this->sql </b><BR><BR>\n";
    echo "DUMP DATA\n";
    echo "<pre>";
    var_dump($this->data);
    echo "RECURSED OBJECT \n\n";
    var_dump($this->recurse);
    echo "</pre>";
  }

Upvotes: 1

kirilloid
kirilloid

Reputation: 14314

AFAIK, it is not possible to insert into several tables with one insert. Moreover, you need to preserve data integrity, so related tables would have right foreign keys.

The general idea is to iterate through the data, insert records and remember inserted ids, then write them as corresponding foreign keys.

You iterate thru your objects, insert all primitive properties as fields, then get an id using mysql_last_insert_id, then while saving offices (or their details) put that id as their related object id.

E.g. we have the following JSON.

{"authors": [
    {"first_name": "John",
     "last_name": "Doe",
     "books": [{
          "title": "Capture the flag",
          "ISBN": "123-456789-12345",
     },{
          "title": "Deathmatch",
          "ISBN": "123-456789-12346",
     }]
]}

Then we insert that data with the following code:

foreach ($data as $author) {
    mysql_query("INSERT INTO `authors` (`first_name`, `last_name`), VALUES('{$author->first_name}', '{$author->last_name}') ");
    $author_id = mysql_last_insert_id();
    foreach ($author->books as $book) {
        mysql_query("INSERT INTO `books` (`title`, `isbn`, `author_id`), VALUES('{$book->title}', '{$book->isbn}', '{$author_id}') ");
    }
}

This is for case you have auto-increment for id's in tables.

Of course, you'll need to validate and escape data before insertion etc.

Upvotes: 4

Related Questions