Why does Perl sometimes recycle references?

Question

The following code:

use strict;
use warnings;

my @array = (0,1,2,3,4,5,6,7,8,9);
print "array ref is ".\@array."\n";

my @copy_refs;
for my $element(@array) {
    my @copy = @array;
    print "copy ref is ".\@copy."\n";
    push @copy_refs, \@copy;
}

produces the following output, as one might expect:

array ref is ARRAY(0x21ae640)
copy ref is ARRAY(0x21e2a00)
copy ref is ARRAY(0x21d7368)
copy ref is ARRAY(0x21d71e8)
copy ref is ARRAY(0x21d70c8)
copy ref is ARRAY(0x21d6fa8)
copy ref is ARRAY(0x21d6e88)
copy ref is ARRAY(0x21d6d68)
copy ref is ARRAY(0x21d6c48)
copy ref is ARRAY(0x21cf8a0)
copy ref is ARRAY(0x21cf780)

However, the same code, with push @copy_refs, \@copy; removed, produces the following output:

array ref is ARRAY(0x229e640)
copy ref is ARRAY(0x22d2a00)
copy ref is ARRAY(0x22d2a00)
copy ref is ARRAY(0x22d2a00)
copy ref is ARRAY(0x22d2a00)
copy ref is ARRAY(0x22d2a00)
copy ref is ARRAY(0x22d2a00)
copy ref is ARRAY(0x22d2a00)
copy ref is ARRAY(0x22d2a00)
copy ref is ARRAY(0x22d2a00)
copy ref is ARRAY(0x22d2a00)

Why is this?

Answer 1

Perl uses reference counting to determine when to free variables. A variable is freed as soon as nothing references it. Code blocks (e.g., subs and files) maintain a reference to the lexical variables declared within them while the variables are in scope, so variables normally get deallocated on scope exit.

In other words, my @copy allocates a new array. When the scope is exited, @copy gets freed if nothing else references it.

When you do push @copy_refs, \@copy;, @copy_refs references the array, so @copy isn't freed.

When you omit push @copy_refs, \@copy;, nothing references @copy, so it's freed. This leaves the memory available to be reused in the next loop pass.

Kinda.

As an optimization, if nothing references a variable when it goes out of scope, it's merely cleared rather than destroyed and recreated. So @copy is not being merely reallocated at the same location; it's actually the very same array that's been cleared.

Answer 2

It's an optimization of Perl's virtual machine. (note that it could be due to "chance" as well, but in that case it's not)

Disclaimer: I don't know the specifics of how Perl optimizes its memory management. However, I am knowledgeable about memory management an virtual machines in general.

A few preliminary words, to be on the same page: Perl uses automatic memory management through a garbage collector (reference counting to be precise). What this (aggressive?) sentence means is that in Perl, you don't need to manually request memory from the OS when you want to create an object, neither do you need to explicitly destroy your object to return memory to the OS.

When you do

my @copy = @array;

Perl allocates an array, and copies @array into it. Naively, allocating an array means asking the OS for some memory, and keeping track that this memory corresponds to the variable @copy. However, asking the OS for memory is very slow.

Furthermore, if every iteration of the for loop was allocating a new array at a new location, it would use a lot of memory: in your case, 10 times the size of @array (because you loop 10 times). This would be bad, especially since at the iteration n+1, you'd have no way to access the @copy allocated at iteration n (because you didn't keep a reference to it).

A visual representation might help understand. Take the following representation of the memory

+-----------------------------------------------------------
+                       empty
+-----------------------------------------------------------

After allocating @copy in the first iteration of the loop, you'll get:

+-----------------------------------------------------------
+  @copy |                     empty
+-----------------------------------------------------------

Now, after the first iteration, there is no way to access @copy anymore. Perl can then destroy it, and your memory could then become:

+-----------------------------------------------------------
+  empty |                   empty 
+-----------------------------------------------------------

And then the second iteration would allocate @copy after the previous one:

+-----------------------------------------------------------
+  empty |  @copy  |                empty
+-----------------------------------------------------------

At the third iteration, you'd get:

+-----------------------------------------------------------
+  empty |  empty  | @copy  |            empty
+-----------------------------------------------------------

And so on for you ten iterations. It's obvious that having those empty spots does no good. This could be optimized in two ways:

• first, when freeing @copy, Perl could see that two empty blocks are next to each other, and merge them. This means that after freeing @copy, you'll go from

  +-----------------------------------------------------------
  +  @copy |                     empty
  +-----------------------------------------------------------

to

  +-----------------------------------------------------------
  +  empty |                     empty
  +-----------------------------------------------------------

to

  +-----------------------------------------------------------
  +                        empty
  +-----------------------------------------------------------

Now, when the next iteration allocates @copy, it will go at the same place the one from the previous iteration was. Thus, you'd get the same reference for all @copy from all iterations.

• Perl could simply see that within your loop, you allocate @copy but don't keep a reference to it, and therefore allocate only once some memory, which will be used for all iterations of @copy. In that case again, you'd get the same reference for all of your @copy.

I do not know if Perl uses either of this mechanism, however, it certainly uses something that produces a similar result (the result being "keeping memory low", which has for consequence "all the @copy from all the iterations have the same address). As per Ikegami's answer, what really happens is something similar to my second bullet point.

---

Note that I've assumed in that answer that Perl doesn't request some new memory from the OS (= malloc) for each object it allocates, but rather requests for bulks of memory (using either mmap directly, or a large malloc). If Perl was using a single malloc for each object it allocates, then it becomes even more important to reuse memory from each @copy: calling malloc and free at each iteration would be very slow.

Answer 3

Ok, my @copy = @array creates new array with name @copy and copies each and every element from @array.

If you push @copy_refs, \@copy then array @copy is not destroyed at the end of the loop cycle step, on new cycle of the loop the @copy is continue to exist, new @copy array is created it occupies new location in memory (memory space address is changed).

If you do not push @copy_refs, \@copy then array @copy is destroyed at the end of loop cycle step and recreated on next cycle step of the loop in probably same memory location (you can see that it's address is the same from previous step of the loop).

Memory occupied by some variable only freed when the code stops to use it - in your case push @copy_refs, \@copy still uses the array and prevents it's destruction (memory is still occupied by the array).

Sorry, probably my explanation is not very intuitive and not very transparent. But the idea is that @copy_refs is keeping location for @copy array which prevents it's destruction.

Answer 4

Probably you intended something of this kind

use strict;
use warnings;

use Data::Dumper;

my $debug = 1;

my @array = (0,1,2,3,4,5,6,7,8,9);

print "array ref is ".\@array."\n";

my @copy_refs;

for my $element(@array) {
    {                                       # new scope opens
        my @copy = @array;                  # new @copy array in new scope
        print "copy ref is ".\@copy."\n";
        push @copy_refs, \@copy;            # due new scope we have new array ref
    }                                       # and here scope closes 
                                            # @copy would be destroyed 
                                            # if it was not stored in @copy_ref
}

print Dumper(\@copy_refs) if $debug;

output ($debug = 0)

array ref is ARRAY(0x2664388)
copy ref is ARRAY(0x26ecb10)
copy ref is ARRAY(0x2663b00)
copy ref is ARRAY(0x2663d40)
copy ref is ARRAY(0x2663f80)
copy ref is ARRAY(0x26641c0)
copy ref is ARRAY(0x2664400)
copy ref is ARRAY(0x2664640)
copy ref is ARRAY(0x26e7250)
copy ref is ARRAY(0x27e5608)
copy ref is ARRAY(0x26889f0)