CODEWITHSUNDEEP
perlfunctionmemory
Lumi
Lumi

Reputation: 15294

Perl: Share subroutine implementation among various packages?

Sort of a fringy, non-practical question coming along here. You know that in Perl you can store a subroutine in a variable, like in Javascript and other languages. A subroutine does take up some memory.

Now you might think that it would save memory to use that one reference to a subroutine to provide the implementation for lots of identical subroutines in various packages. Like all those setters and getters. (Yes, I should use one of the existing mooses or mice or whatnot, but this is a non-practical question.) Or like other routines you might want to have in various packages, to enable all those packages to do some cross-cutting wtf.

You could have a delegate in each class to do the job. But maybe that's too heavy and too OO. You want the subroutine. Maybe this is called mix-in? Anyway you want to install that one subroutine in every package, thinking that it saves memory, and you can test it one line, like this:

perl -lwe "$s=sub { print 33 };
  for (1 .. 20000) { $p=qq(Bla${_}::bla); *$p = $s} Bla987::bla(); <>"

(Okay, it went on two lines to accommodate page width here. And note that double quotes are required on Windows, and on Linux you'd use single quotes to avoid having the shell eat your dollars.)

Now what this funny piece of code (of dubious quality, just for demonstration purposes) does is creating 20,000 packages and installing that subroutine in each package (in the corresponding symbol table entry).

I can see memory consumption going up and down in {{taskmgr}} according to the number of packages I create (12.2 KB for 20,000 packages, 6.7 for 10,000 packages, 1.0 KB for 100 packages - not much, all in all), but I guess this is not because of the sub getting copied, but rather because of the extra packages being created.

Is that correct? The sub isn't copied, it's only referenced?

What's good reading regarding this topic? Or is this venturing into implementation details and perl5porters stuff?

Or is the entire train of thought here just misguided musing and wondering?

Update

Okay, I'm not sure the {{sub}} in {{$s}} gets reused - because when I create new subroutines in a loop memory usage does not go up:

perl -lwe "for (1 .. 20000) {
  $p=qq(Bla${_}::bla); *$p = sub {print 33} } Bla987::bla(); <>"

But maybe that's because the sub is constant.

Memory usage does go up (to 10 KB for 10,000 packages and 20 KB for 20,000) if I introduce a variant element in the sub, like this:

perl -lwe "for my $i (1 .. 10000) {
  $p=qq(Bla${i}::bla); *$p = sub {print $i} } Bla987::bla(); <>"

Upvotes: 2

Views: 212

Answers (1)

Eric Strom
Eric Strom

Reputation: 40152

Perl will only compile the subroutine once. You are passing around the reference to it, which will be the same if you check the addresses.

The memory leak comes from creating 20,000 packages, which are 20,000 hash references, each with some magic that takes a bit more memory. Each of those hash references contains a copy of the string with the subroutine name. And a reference to each of those hashes is placed into the main:: symbol table.

If you want a subroutine to be in scope in all packages, use a fully qualified name, place the subroutine into a lexical that spans all of the packages, or put the subroutine into a symbol variable:

 sub some::name {}  # call as `some::name()` from anywhere

 my $x = sub {};    # call as `$x->()` anywhere $x is in scope

 *!    = sub {};    # call as `&!()` from anywhere

Either of the solutions above will give you access from other packages without wasting lots of memory.

Per your update, you are seeing a size increase with the closures because while each subref is the same, each has to have a new lexical pad created to hold the closure variable, which will be unique for each closure.

You can also make methods available everywhere using the UNIVERSAL package, but it can be problematic, so take the following with a grain of salt:

sub UNIVERSAL::foo {say "foo(@_)"}

xyz->foo;  #  foo(xyz)

Upvotes: 5

Related Questions