XML to MySQL when xml file has multiple matching fields

Question

I've been doing some work on an XML to Mysql using load XML. I have been successful with itin the past. The difference with the latest effort is that we have multiple occurrences of a field-name in the MySQL. A sample of this is below:

<row>
<pictures>
        <picture name="Photo 1">
          <filename>image1.jpg</filename>
        </picture>
        <picture name="Photo 2">
          <filename>image2.jpg</filename>
        </picture>
        <picture name="Photo 4">
          <filename>image3.jpg</filename>
        </picture>
        <picture name="Photo 3">
          <filename>image4.jpg</filename>
        </picture>
        <picture name="Photo 7">
          <filename>image5.jpg</filename>
        </picture>
        <picture name="Photo 6">
          <filename>image6.jpg</filename>
        </picture>
        <picture name="Photo 5">
          <filename>image7.jpg</filename>
        </picture>
        <picture name="Photo 8">
          <filename>image8.jpg</filename>
        </picture>
        <picture name="Photo 9">
          <filename>image9.jpg</filename>
        </picture>
      </pictures>
</row>

I need to import this into a MySQL table with the fields:

• picture1
• picture2
• picture3
• picture4
• picture5
• picture6
• picture7
• picture8
• picture9

As you can see, the 'name' attribute doesn't necessarily occur in the correct order, so I need them to simply be inserted in order. So the first <filename> to go to picture1, the second <filename> to picture2 etc..

What is currently being achieved is that I always end up with the last <picture> entry in the list being in the table. This is I assume because the filed is being overwritten each time.

Any ideas how to achieve this? I have found similar queries to this but no answers as yet and have been looking for a good while. The rest of the file is loading fine as they have unique field-names and can easily be mapped to a MySQL column, but I am struggling with this one.

Answer 1

And here is the XSLT solution. As information, XSLT is a declarative special-purpose language to transform, re-style, and restructure XML documents in various formats for end use purposes. PHP maintains an XSLT processor. Be sure to uncomment out extension=php_xsl.dll

XLST (accommodates image numbers greater than two digits)

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output version="1.0" encoding="UTF-8"/>

<xsl:template name="picturesort" match="pictures" >  
  <row>  
   <pictures>   
     <xsl:for-each select="picture">    
        <xsl:variable name="numkey" 
            select="substring-after(substring-before(filename, '.'), 'e')"/>            
        <picture name="{../picture[substring-after(@name, ' ') = $numkey]/@name}"> 

            <xsl:copy-of select="filename"/> 
        </picture>   
      </xsl:for-each>  
    </pictures>  
  </row>   
</xsl:template>

</xsl:stylesheet>

XML OUTPUT

 <?xml version="1.0" encoding="UTF-8"?>

<row>
<pictures>

<picture name="Photo 1">
<filename>image1.jpg</filename>
</picture>

<picture name="Photo 2">
<filename>image2.jpg</filename>
</picture>

<picture name="Photo 3">
<filename>image3.jpg</filename>
</picture>

<picture name="Photo 4">
<filename>image4.jpg</filename>
</picture>

<picture name="Photo 5">
<filename>image5.jpg</filename>
</picture>

<picture name="Photo 6">
<filename>image6.jpg</filename>
</picture>

<picture name="Photo 7">
<filename>image7.jpg</filename>
</picture>

<picture name="Photo 8">
<filename>image8.jpg</filename>
</picture>

<picture name="Photo 9">
<filename>image9.jpg</filename>
</picture>

</pictures>
</row>

PHP

<?php

// Load the XML source
$xml = new DOMDocument;
$xml->load('C:/Path/To/XMLfile.xml');

$xsl = new DOMDocument;
$xsl->load('C:/Path/To/XSLfile.xsl');

// Configure the transformer
$proc = new XSLTProcessor;
$proc->importStyleSheet($xsl);

// Transform XML source
$newXml = $proc->transformToXML($xml);
echo $newXml;

// Save output to file
file_put_contents("C:/Path/To/NewXMLfile.xml", $newXml);

?>

Answer 2

As the XML does not match the format you aim for you need to transform it first. Traditionally this is done with XSLT but you can also do this with XMLReader and XMLWriter in PHP which has the benefit that it does not require to keep the whole XML document(s) in memory.

The XMLReaderIterator package has support for such operations, an example is already given with the library.

Creating a modification of that example code by taking your specific case and an exemplary input file named pictures.xml and keeping the output to the standard-output for demonstration purposes allows me to quote the following excerpt:

[... starts like examples/read-write.php] 

/** @var $iterator XMLWritingIteration|XMLReaderNode[] */
$iterator = new XMLWritingIteration($writer, $reader);
$writer->startDocument();

$rename = ['row' => 'resultset', 'pictures' => 'row'];

$trimLevel    = null;
$pictureCount = null;

foreach ($iterator as $node) {
    $name         = $node->name;
    $isElement    = $node->nodeType === XMLReader::ELEMENT;
    $isEndElement = $node->nodeType === XMLReader::END_ELEMENT;
    $isWhitespace = $node->nodeType === XMLReader::SIGNIFICANT_WHITESPACE;

    if (($isElement || $isEndElement) && $name === 'filename') {
        // drop <filename> opening and closing tags
    } elseif ($isElement && $name === 'picture') {
        $writer->startElement('field');
        $writer->writeAttribute('name', sprintf('picture%d', ++$pictureCount));
        $trimLevel = $node->depth;
    } elseif ($trimLevel && $isWhitespace && $node->depth > $trimLevel) {
        // drop (trim) SIGNIFICANT_WHITESPACE
    } elseif ($isElement && isset($rename[$name])) {
        $writer->startElement($rename[$name]);
        if ($rename[$name] === 'row') {
            $pictureCount = 0;
        }
    } else {
        $iterator->write();
    }
}

This is one XMLWritingIteration that is composed of an XMLReader and XMLWriter object. That iteration allows you to take over everything from the input document (via $iterator->write()) and do the needed changes only on occasions:

• drop the <filename> and </filename> tags
• create <field> elements with the correct name attributes to have the pictures in document order (Mysql XML nomenclature)
• drop significant whitespace as <filename> tags are dropped as well
• rename the document element from <row> to <resultset> (Mysql XML nomenclature)
• rename the <pictures> element to <row> (again Mysql XML nomenclature)
• the counter for the picture fields is reset per each (output) row
• everything else is kept as-is

Such a transformation results in the following example output with the XML presented in your question:

<?xml version="1.0"?>
<resultset>
    <row>
        <field name="picture1">image1.jpg</field>
        <field name="picture2">image2.jpg</field>
        <field name="picture3">image3.jpg</field>
        <field name="picture4">image4.jpg</field>
        <field name="picture5">image5.jpg</field>
        <field name="picture6">image6.jpg</field>
        <field name="picture7">image7.jpg</field>
        <field name="picture8">image8.jpg</field>
        <field name="picture9">image9.jpg</field>
    </row>
</resultset>

For more information about the XML format used by Mysql, please see the Mysql documentation for the --xml commandline switch which describes the standard XML output format which can be read in by LOAD XML.

For this little example you could as well use XSLT as there would be no problem to do the whole transformation in memory. But if you need to look for memory (which can happen if you deal with XML database dumps), the XMLWritingIteration allows iteration based XML transformation with an XML Pull parser (XMLReader) and forward-only XML output via XMLWriter.

Answer 3

Possible way:

• iterate over all <picture> in a <row>
• build an associative array with key = name and value = filename
• sort array by keys
• feed the array to your DB