CODEWITHSUNDEEP
perlgarbage-collection
Elad
Elad

Reputation: 11

How to release memory in Perl?

I wrote Perl code that handle with xml files and create array from the records of this xml file. When the array is large then I got 'Out of memory' error while I over it or use 'join' on it.

There is any way to release memory on Perl?

I get **$data** to the function:

my @records = ();
($records_section) = $data =~ /<gift-doc_body>(.+)<\/gift-doc_body>/ms;

(@records) = $records_section =~ /<gift-doc_document>(.+?)<\/gift-doc_document>/msg;

$new_xml = join("\n", "<root>", @records, "</root>");

I got the 'Out of memory' error when it doing the 'join'

Upvotes: 0

Views: 465

Answers (2)

Sobrique
Sobrique

Reputation: 53488

Perl does its own garbage collection. You don't need to worry about freeing memory - it does it automatically when you stop the program execution.

The way you 'free' it is by stopping referencing it. So if you keep scopes as tight as possible you don't waste memory particularly.

However XML is a bit of a special case - the problem with XML is that a) its memory footprint is about 10x the file size and b) it has to have matched tags, which means you can end up having to parse the whole file to ensure the tags are matched.

It's likely that this is the problem you're having - a large XML file. To handle a large XML file, you can't really read the whole thing - which you need to to ensure it's valid.

However, once of the parsing libraries - XML::Twig allows you to use handlers to parse subsets of the file as you go. You should consider doing this. Look at purge which will release the memory as you go: 

#!/usr/bin/perl

use strict;
use warnings;

use XML::Twig;

sub process_some_element {
    my ( $twig, $some_element ) = @_;
    $some_element->print;
    $twig->purge;
}

my $twig =
    XML::Twig->new(
    twig_handlers => 
        { 'some_element' => \&process_some_element } );
$twig->parsefile('sample.xml');

The important part here is the purge - because it discards XML seen thus far. You can also use flush which does much the same thing, but prints the 'seen' XML, if - for example - you want to modify and retain the document structure.

Upvotes: 5

Eitan
Eitan

Reputation: 128

First - Perl doesn't "free" memory - it has GC for that. Second of all, when you read to memory, it's better not to read all (as stated by Nitek). I suggest you use XML::Twig or XML::XPath to read and parse the file as it handles the "big" data processing for you (in chunks).

Upvotes: 4

Related Questions