CODEWITHSUNDEEP
xmlperlxml-twig
Kenny
Kenny

Reputation: 1090

XML::Twig: is parsefile() more efficient than parse()?

I've written a fairly basic webapp in Perl and it processes XML files using XML::Twig. These XML files are fairly large and complex, so I'm deliberately using the chunk by chunk varient of XML::Twig rather than the "load it all at once" method.

However, this webapp completely crashes and dies if I load in a large XML document, even when using the chunking method. I couldn't get any clues as to why this was happening, because the webapp is hosted on a shared server with 1and1.co.uk, and I can't see the Apache error log file. Even when wrapping the calls in an eval{} block, I couldn't catch it from dying. Annoyingly, it works fine on my development server at home, so I couldn't reproduce the problem.

To get it working, I made a change so that instead of using the parse() method and passing in a scalar containing the entire XML, I wrote the XML to a file, then used parsefile( $filename ) instead. When I made that change, it worked.

I'm just a bit confused by this, tbh, and I attempted to find out via the wonder of Google if parsefile() is indeed more efficient that parse(), but failed to locate anything. Does anyone happen to know?

Upvotes: 1

Views: 890

Answers (2)

Borodin
Borodin

Reputation: 126742

I am sure 1 and 1 allow you access to the Apache log file as it is a vital tool for debugging CGI and web apps such as yours. Give them a shout and ask them how to do it.

If your XML files are large then it spoils the point of using XML::Twig in chunk mode. It also seems likely that your app is failing on the server because it has exceeded its memory quota. Again, a call to your web hosting company will tell you whether this is the case.

How does your XML get into memory in the first place? If you are slurping it into memory from an XML file then just leave the fix as it is and get XML::Twig to read directly from the file. If you are fetching the XML from a remote URL then remember that XML::Twig has a parseurl method which will avoid fetching the data to a local file. I can't think of another likely source so you will have to explain.

Upvotes: -1

brian d foy
brian d foy

Reputation: 132863

Look at the source. They are the same thing.

In XML::Twig parsefile is just an extended XML::Parser::parsefile (the superclass):

sub parsefile
  { my $t= shift;
    if( -f $_[0] && ! -s $_[0]) { return _checked_parse_result( undef, "empty file '$_[0]'"); }
    $t= eval { $t->SUPER::parsefile( @_); };
    return _checked_parse_result( $t, $@);
  }

In XML::Parser, parsefile is just a wrapper around parse:

sub parsefile {
  my $self = shift;
  my $file = shift;
  local(*FILE);
  open(FILE, $file) or  croak "Couldn't open $file:\n$!";
  binmode(FILE);
  my @ret;
  my $ret;

  $self->{Base} = $file;

  if (wantarray) {
    eval {
      @ret = $self->parse(*FILE, @_);
    };
  }
  else {
    eval {
      $ret = $self->parse(*FILE, @_);
    };
  }
  my $err = $@;
  close(FILE);
  die $err if $err;

  return unless defined wantarray;
  return wantarray ? @ret : $ret;
}

Upvotes: 2

Related Questions