Weird Powershell performance issue

Question

NOTE: This is a solution for Project Euler Problem 14. If you still want to solve it yourself then don't read on.

The problem was to find the number below one million which, as starting number for the Collatz sequence, produces the longest such sequence. My initial code was as follows:

$r = @{}

for($i = 1; $i -lt 1000000; $i++) {
    $n = 0
    $a = $i
    while ($a -gt 1) {
        if ($r[$a]) {
            $n += $r[$a]
            break
        }
        if ($a % 2 -eq 0) {
            $a /= 2
        } else {
            $a = $a * 3 + 1
        }
        $n++
    }
    $r[$i] = $n
}

$r.GetEnumerator() | sort Value | select -l 1 | %{$_.Key}

which tries to use a hashtable as cache for sub-sequences already encountered to save time. I was quite surprised when that script had a runtime of over eight minutes on my machine. Re-creating the same code in C#:

using System;
using System.Collections.Generic;

class Problem14
{
    public static void Main()
    {
        var ht = new Dictionary<long, int>();

        for (int i = 1; i < 1000000; i++)
        {
            int count = 0;
            long j = i;

            while (j > 1)
            {
                if (ht.ContainsKey(j))
                {
                    count += ht[j];
                    break;
                }
                if (j % 2 == 0)
                    j /= 2;
                else
                    j = 3 * j + 1;

                count++;
            }
            ht[i] = count;
        }

        KeyValuePair<long, int> max = new KeyValuePair<long, int>();
        foreach (var n in ht)
        {
            if (n.Value > max.Value)
                max = n;
        }
        Console.WriteLine(max.Key);
    }
}

had a runtime of just over one second. I knew that speed of execution wasn't a prime goal in Powershell. It's an adminstration language and for those tasks the ratio of PS code to cmdlets is likely very different from what I do here.

Still, I don't know what exactly causes the slowdown.

Suspecting the hash table, I replaced it for caching with an array. This resulted in an execution time around 200 ms in C# and about 32 minutes in Powershell. Code was as follows:

$r = ,0*1000000

for($i = 1; $i -lt 1000000; $i++) {
    $n = 0
    $a = $i
    while ($a -gt 1) {
        if ($r[$a]) {
            $n += $r[$a]
            break
        }
        if ($a % 2 -eq 0) {
            $a /= 2
        } else {
            $a = $a * 3 + 1
        }
        $n++
    }
    if ($i -lt 1000000) {
        $r[$i] = $n
    }
}

$max = 0
for($i=1; $i -lt 1000000; $i++) {
    if ($r[$i] > $r[$max]) {
        $max = $i
    }
}
$max

and

using System;

class Problem14
{
    public static void Main()
    {
        var cache = new int[1000000];

        for (int i = 1; i < 1000000; i++)
        {
            int count = 0;
            long j = i;

            while (j > 1)
            {
                if (j < 1000000 && cache[j] != 0)
                {
                    count += cache[j];
                    break;
                }
                if (j % 2 == 0)
                    j /= 2;
                else
                    j = 3 * j + 1;

                count++;
            }
            cache[i] = count;
        }

        var max = 0;
        for (int i = 1; i < cache.Length; i++)
        {
            if (cache[i] > cache[max])
                max = i;
        }

        Console.WriteLine(max);
    }
}

A completely cacheless variant was around 1.2 seconds in C#. Didn't try yet in Powershell.

Any ideas?

Answer 1

First and foremost, PowerShell is an interpreted language (not jitted, nor compiled). That's always going to hurt. ;-)

Secondly, there are some language constructs that you may use that can avoid multiple interpreted steps. For example, instead of using a for(;;) statement, use a range:

0..1000000 | foreach-object { foo $_ 

Might help a bit.

Most importantly, avoid the excessive use of break and continue keywords in loops - invert your logic if possible to avoid this. Internally, these keywords signal using .NET exceptions, and are therefore expensive operations.

Hope this helps your understanding somewhat.

EDIT: these keywords use .NET exceptions for signalling

From System.Management.Automation.FlowControlNode.Execute(...):

switch (this._tokenId)
    {
        case TokenId.ExitToken:
        {
            int exitCode = this.GetExitCode(result);
            throw new ExitException(base.NodeToken, exitCode);
        }
        case TokenId.ReturnToken:
            throw new ReturnException(base.NodeToken, result);

        case TokenId.BreakToken:
            label = this.GetLabel(result, context);
            throw new BreakException(base.NodeToken, label);

        case TokenId.ContinueToken:
            label = this.GetLabel(result, context);
            throw new ContinueException(base.NodeToken, label);

-Oisin