CODEWITHSUNDEEP
phpxmlsimplexmlxml-parsing
bahamut100
bahamut100

Reputation: 1837

Memory limit and very big XML file

I need to parse a very big XML file, with a filesize of 750Mo !

I have meomy limit at 512M

ini_set('memory_limit', '512M');

I have no problem to open file under 30Mo, but with 750Mo, I obtain a fatal error

Fatal error: Allowed memory size of 1677721600 bytes exhausted (tried to allocate 2988843769 bytes)

I do that to open files :

$fichier = file_get_contents($inputfileName);
$xmlInput = simplexml_load_string(utf8_encode($fichier));

Have you an idea to open this file ?

Upvotes: 4

Views: 9809

Answers (3)

user2845163
user2845163

Reputation: 1

For big file, perfect use XMLReader class. But if liked simplexml:

Code: https://github.com/dkrnl/SimpleXMLReader/blob/master/library/SimpleXMLReader.php

Usage example: http://github.com/dkrnl/SimpleXMLReader/blob/master/examples/example1.php

Upvotes: 0

Gordon
Gordon

Reputation: 317177

Using the DOM based extensions will take up significantly more memory as the raw XML is because the XML will be parsed completely into a tree structure of nodes. Have a look at XMLReader instead

The XMLReader extension is an XML Pull parser. The reader acts as a cursor going forward on the document stream and stopping at each node on the way.

and make sure you parse with LIBXML_PARSEHUGE

An alternative would the event-based XMLParser

Upvotes: 6

phs
phs

Reputation: 11061

You want a SAX or other event-based xml parser. Google 'php sax parser'.

Upvotes: 0

Related Questions