powershell Compress-Archive OutOfMemoryException

Question

I need to compress a folder with power shell.

There is my code:

Get-ChildItem $YourDirToCompress -Directory  | 
           where { $_.Name -notin $DirToExclude} | 
              Compress-Archive -DestinationPath $ZipFileResult -Update

Move-Item -Path $ZipFileResult -Destination $ZipFileDest

I get:

Exception calling "Write" with "3" argument(s): "Exception of type 'System.OutOfMemoryException' was thrown."
At 
C:\Windows\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:820 
char:29
+ ...                     $destStream.Write($buffer, 0, $numberOfBytesRead)
+                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : OutOfMemoryException

I have set :

Set-Item WSMan:\localhost\Plugin\Microsoft.PowerShell\Quotas\MaxMemoryPerShellMB 8000
Set-Item WSMan:\localhost\Shell\MaxMemoryPerShellMB 8000
Restart-Service WinRM

The whole file is about 1.9 GiB and the compressed file is 500 MiB. I find it hard to believe it is really a memory problem.

Also, once or twice it succeeded on file creation (when MaxMemoryPerShellMB was set to 4000). But most times it fails.

What can I do?

Answer 1

It seems that the built-in Zip module of PowerShell is quite limited and designed for simple straightforward tasks. One of those limitations is that the max file size is 2GB. Also, the way that it obtains memory seems not very efficient. For example, if you try to zip files by Winzip or WinRar using Windows UI, it will probably work. More info about the underlying library https://learn.microsoft.com/en-us/dotnet/api/system.io.compression.ziparchive

So, I recommend using 7Zip4Powershell library which worked perfectly for me with the same files/ Windows memory etc.

Answer 2

I was getting the exact same error in 2022:.

Exception calling "Write" with "3" argument(s): "Exception of type
'System.OutOfMemoryException' was thrown."
At C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Arch
ive\Microsoft.PowerShell.Archive.psm1:820 char:29
+ ...                     $destStream.Write($buffer, 0, $numberOfBytesRead)
+                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : OutOfMemoryException

OutOfMemoryException Microsoft.PowerShell.Archive.psm1

I was able to resolve it very simply after a bit of thinking.

I too doubted that it was really a memory problem, since I have 16 GiB of RAM and it was showing only around 9 GiB in use.

Microsoft Scripting Guy Ed Wilson has written an article Learn How to Configure PowerShell Memory, and for some this may provide the solution. In my case it did not, as PS appeared to be already configured to use the maximum available memory.

I was getting the issue on a folder I normally zip up every few days. This folder is over 11 GiB and zips down to around 4 GiB. This had previously been working for years.

However, thinking about it, to create a zip of this size I think it's likely that PS holds a significant amount of data in memory, more and more right up to the point when it commits it to the file system at the end. And I realized that although the computer has 16 GiB of RAM and was showing only around 9 GiB in use, 9 GiB is an unusually large amount of memory to be in use on that computer. Subtracting memory used by Windows, graphics etc still left several GiB free—but perhaps not enough for PS to create a large zipfile.

Sure enough, after a reboot to release the unusually large amount of RAM that had been in use, the process ran successfully and created my zipfile as usual.

In summary:

• Check whether you can configure PS to use more RAM;
• Check in Task Manager to see if you have less RAM free than usual;
• Reboot to make maximum RAM available to PS.