PowerShell remote invocation mysteriously hangs

Question

I have created a series of functions that basically collect all the IIS configurations about a site, when run on a server locally it executes without issue (albeit slowly) however when I run them remotely using an invoke-command in PowerShell 2 it runs through and mysteriously stops approximately 15-20 seconds into the process. It generally stalls on the same request but not always. The same commands executed locally work without any issues. No exception is raised, it just hangs indefinitely.

I can post the code if necessary however it is several hundred lines so I'm more looking for guidance on how to investigate a problem like this or if anyone has encountered something similar.

Comparing IISConfig between [targetserver] and localhost.
Checking Installed IIS version on [targetserver]:
IIS major version : 7
IIS minor version : 5
IIS7+ detected, using WebAdmin module and IIS metabase

Name                                                        Value
----                                                        -----
name                                                        Default Web Site
id                                                          1
serverAutoStart                                             True
state                                                       1


Site Configuration:

Name        Path        PSPath      Handlers_Ac Access_sslF Asp_AppAllo Asp_AppAllo Asp_limits_ Asp_EnableP Asp_limits_
                                    cessFlags   lags        wClientDebu  wDebugging bufferingLi  arentPaths queueTimeou
                                                                      g                     mit             t
----        ----        ------      ----------- ----------- ----------- ----------- ----------- ----------- -----------
Default ... IIS:Site... WebAdmin... Read,Script                   False       False    25000000        True 00:00:00


WebApp VDir: /MyApp, App Pool: MyApp
App pool Configuration:

AppPoolID   Enable32Bit managedPipe managedRunt AppPoolName AppPoolAuto processMode processMode processMode recycling_l
             AppOnWin64 lineMode    imeVersion                    Start l_idleTimeo l_identityT l_UserName  ogEventOnRe
                                                                        ut          ype                     cycle
---------   ----------- ----------- ----------- ----------- ----------- ----------- ----------- ----------- -----------
                  False Classic     v2.0        MyApp                True 00:20:00    LocalSer...             Time,Req...


Analyzing web directories for /MyApp, this could take a while....
Initial Collection Completed, found 141... took 0.9516122 seconds
0 C:\inetpub\wwwroot\MyApp\Core
1 C:\inetpub\wwwroot\MyApp\Core\AdminTools
2 C:\inetpub\wwwroot\MyApp\Core\AdminTools\Cache
3 C:\inetpub\wwwroot\MyApp\Core\AdminTools\Extra
4 C:\inetpub\wwwroot\MyApp\Core\AdminTools\HTTPPostTest
5 C:\inetpub\wwwroot\MyApp\Core\AdminTools\IISAdmin
6 C:\inetpub\wwwroot\MyApp\Core\AdminTools\Profiling
7 C:\inetpub\wwwroot\MyApp\Core\AdminTools\RecordTestData
8 C:\inetpub\wwwroot\MyApp\Core\AdminTools\ScrambleTest
9 C:\inetpub\wwwroot\MyApp\Core\AdminTools\Sessions
Analyzed 10 so far... took 6.7236862 seconds, remaining time 88.08028922 seconds
Current Folder: C:\inetpub\wwwroot\MyApp\Core\AdminTools\Sessions
10 C:\inetpub\wwwroot\MyApp\Core\AdminTools\SoapTest
11 C:\inetpub\wwwroot\MyApp\Core\AdminTools\StaticContent

Sometimes it makes it to 15 or so. I tried from my laptop and from one server to another and the behavior is the same.

Here is the loop which is hanging:

$start = [System.DateTime]::Now
$numanalyzed = 0
if ($true) #skip to test
{
# loop through all physical folders as it is much faster
foreach ($folder in $folders)
{

    write-host $numanalyzed $folder.fullname
    #figure out the virtual path to the folder
    $iis7vwebfolderpath = $folder.FullName.Replace($iis7webapp.PhysicalPath, $iis7VDirWebApppath)
    #Get-item $iis7vwebfolderpath | gm
    $iis7VWebDirConfigItem = Get-LNOSIIS7ConfigForPSPath -PSPath $iis7vwebfolderpath

    # add new item to list
    $iis7VWebDirConfig += $iis7VWebDirConfigItem

    # increment counter and report out progress every 10
    $numAnalyzed++
    if ($numanalyzed % 10 -eq 0) 
    {

       $end = [System.DateTime]::Now
       $timeSoFar = (NEW-TIMESPAN –Start $Start –End $End).TotalSeconds
       $timeremaining = ($folders.Count - $numAnalyzed) * ($timeSoFar / $numanalyzed)
       "Analyzed {0} so far... took {1} seconds, remaining time {2} seconds" -f $numanalyzed,$timeSoFar,$timeremaining  | write-host 
       "Current Folder: {0}" -f $folder.FullName | Write-Host
    }
}
}



$end = [System.DateTime]::Now
"Processed web dirs: {0} took {1} seconds" -f $iis7VWebDirConfig.Count,(NEW-TIMESPAN –Start $Start –End $End).TotalSeconds | write-host   | Write-Host

The function I'm having performance problems with and I've got a separate question about but this post has the source code for the function:

web-administration vs WMI to query web directory properties performance problems

Answer 1

As an FYI if Invoke-Command always hangs:

1. Try a simple command to system : Invoke-Command -ComputerName XXXXX -ScriptBlock { Get-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion }

2. Start the Windows Remote Management Service (on that system)

3. Check for the listening port:

    netstat -aon | findstr "5985"
         TCP    0.0.0.0:5985           0.0.0.0:0              LISTENING       4
         TCP    [::]:5985              [::]:0                 LISTENING       4
   

Answer 2

In my case, it seemed my PowerShell call froze due to the Idle-Timeout expiration (the call runs for a very long time).

Setting IdleTimeout value to a sufficiently long duration fixed my issue.

Once again, query the current configuration using

winrm get winrm/config/winrs

And set the timeout using

winrm set winrm/config/winrs '@{IdleTimeout="18000000"}'

Answer 3

I think i may have discovered the problem, i started getting some odd failures in other parts of the script:

 [SEVERNAME] Processing data from remote server SERVERNAME failed with the following error message: The WSMan provider host process did not return a proper response.  A provider in the host process may have behaved improperly. For more information, see the about_Remote_Troubleshooting Help topic. 
 + CategoryInfo          : OpenError: (SERVERNAME:String) [], PSRemotingTransportException
 + FullyQualifiedErrorId : 1726,PSSessionStateBroken

and

Processing data for a remote command failed with the following error message: Not enough storage is available to complete this operation. For more information, see the about_Remote_Troubleshooting Help topic.
+ CategoryInfo          : OperationStopped (System.Manageme...pressionSyncJob:PSInvokeExpressionSyncJob) [], PSRemotingTransportException
+ FullyQualifiedErrorId : JobFailure

This lead me to the following site: http://www.gsx.com/blog/bid/83018/Troubleshooting-unknown-PowerShell-error-messages

The following recommendations seems to have cleared up most of the problems although i still have some testing to do.

Excerpt from site below:

As the first error message specifies, an overflow of memory in the remote session has occurred. Open a PowerShell prompt on the remote server and display the configuration of winrs using:

winrm get winrm/config/winrs

Check the "MaxMemoryPerShellMB" value. It is set by default to 150 MB on Windows Server 2008 R2 and Windows 7. This is something that Microsoft changed in Windows Server 2012 and Windows 8 to 1024 MB.

In order to resolve this issue, you need to increase the value to at least 512 MB with the following command:

winrm set winrm/config/winrs `@`{MaxMemoryPerShellMB=`"512`"`}