CODEWITHSUNDEEP
perliteration
Oleg V. Volkov
Oleg V. Volkov

Reputation: 22421

Does perl optimize foreach over a range?

Given a simple program:

use strict;
use warnings;

my $count = 0;

foreach ('a' .. 'zzzzzzzzzz') {
    $count++;
}

print "$count\n";

Will perl create entire range of about 1.4+e14 elements in memory and then iterate over them or does it have any internal optimizations to simply step over one state of range operator at a time?

Well, I've actually ran that, so empirical data from top shows it is the latter (and it didn't finish yet to see data from time as well), but is it documented anywhere or where to find confirmation for that in sources?

Upvotes: 4

Views: 629

Answers (2)

ikegami
ikegami

Reputation: 385645

Yes, the following is optimized into a counting loop:

 for (EXPRX..EXPRY) { ... }

perlop mentions this.

In the current implementation, no temporary array is created when the range operator is used as the expression in foreach loops, but [versions of Perl older than 5.6] might burn a lot of memory when you write something like this:

for (1 .. 1_000_000) {
    # code
}

You can see this using the following:

$ perl -e'
   my $n = 10_000_000;
   system("ps h -o rss $$");
   for (1..$n) { system("ps h -o rss $$"); last; }
'
1876
1944

Even within the loop, there's only 1944 KiB used. This is the same amount of that was allocated before the loop, and nowhere near enough to hold 10,000,000 scalars. On the other hand, the following slightly different code shows 385 MiB being used in the loop:

$ perl -e'
   my $n = 10_000_000;
   system("ps h -o rss $$");
   for ((), 1..$n) { system("ps h -o rss $$"); last; }
'
 1876
394724

In this latter version, all 10 million scalars are allocated.

In fact, it's not just EXPR..EXPR that's special. All of the following are implemented differently:

  • for (EXPR; EXPR; EXPR) ("C-style for loop", an augmented while loop.)
    Functionally different.
  • for (EXPRX..EXPRY) (A range and nothing else.)
    Incrementing counting loop.
  • for (reverse CONSTX..CONSTY) (A constant range, preceded by reverse.)
    Descending counting loop over indexes of array built at compile-time.[1]
  • for (reverse EXPRX..EXPRY) (A variable range, preceded by reverse.)
    Descending counting loop.
  • for (@ARRAY) (An array and nothing else.)
    Counting loop over indexes of array.
  • for (reverse @ARRAY) (Reverse of an array and nothing else.)
    Descending counting loop over indexes of array.
  • for (reverse LIST) (Any list that doesn't fit the above patterns.)
    Flattened list iterated from end to start.
  • for (LIST) (Any list that doesn't fit the above patterns.)
    Flattened list iterated from start to end.

  1. Everywhere but in for (CONSTX..CONSTY), 

    CONSTX..CONSTY

    is optimized to

    my @anon;
BEGIN { @anon = CONSTX..CONSTY; }
@anon

    You can see this here:

    $ perl -e'
   BEGIN { system("ps h -o rss $$"); }
   system("ps h -o rss $$");
   exit(0);
   for (reverse 1..10_000_000) {  }
'
 1868
709540

Upvotes: 6

Botje
Botje

Reputation: 30832

From perldoc perlop:

In the current implementation, no temporary array is created when the range operator is used as the expression in foreach loops, but older versions of Perl might burn a lot of memory when you write something like [ what you wrote ]

Upvotes: 7

Related Questions