Exclude list in PowerShell Copy-Item does not appear to be working

Question

I have the following snippet of PowerShell script:

$source = 'd:\t1\*'
$dest = 'd:\t2'
$exclude = @('*.pdb','*.config')
Copy-Item $source $dest -Recurse -Force -Exclude $exclude

Which works to copy all files and folders from t1 to t2, but it only excludes the exclude list in the "root"/"first-level" folder and not in sub-folders.

How do I make it exclude the exclude list in all folders?

Answer 1

I wrote this for daily use and packaged it in the script module, it maintains all the directory structure and supports wildcards:

function Copy-Folder {
    [CmdletBinding()]
    param(
        [Parameter(Mandatory)]
        [String]$FromPath,

        [Parameter(Mandatory)]
        [String]$ToPath,

        [string[]] $Exclude
    )

    if (Test-Path $FromPath -PathType Container) {
        New-Item $ToPath -ItemType Directory -ErrorAction SilentlyContinue | Out-Null
        Get-ChildItem $FromPath -Force | ForEach-Object {
            # avoid the nested pipeline variable
            $item = $_
            $target_path = Join-Path $ToPath $item.Name
            if (($Exclude | ForEach-Object { $item.Name -like $_ }) -notcontains $true) {
                if (Test-Path $target_path) { Remove-Item $target_path -Recurse -Force }
                Copy-Item $item.FullName $target_path
                Copy-Folder -FromPath $item.FullName $target_path $Exclude
            }
        }
    }
}

Just call the Copy-Folder -FromPath 'fromDir' -ToPath 'destDir' -Exclude *.pdb,*.config

The -FromPath and -ToPath can be omitted,

Copy-Folder -FromPath 'fromDir destDir -Exclude *.pdb,*.config

Answer 2

One way of copying items from one folder to another using regular expressions for exclusion:

$source = '.\source'
$destination = '.\destination'
$exclude = '.*\.pdf$|.*\.mp4$|\\folder1(\\|$)|\\folder2(\\|$)'

$itemsToCopy = Get-ChildItem $source -Recurse |
    Where-Object FullName -notmatch $exclude | Select-Object -Expand FullName

$sourceFullNameLength = (Get-Item $source).FullName.Length

foreach ($item in $itemsToCopy) {
    $relativeName = $item.Substring($sourceFullNameLength + 1)
    Copy-Item -Path $item -Destination "$destination\$relativeName"
}

Answer 3

The below snippet will copy all files and folders from $source to $dest, excluding .pdb and .config files from the root folder and sub-folders:

Get-ChildItem -Path $source | Copy-Item -Destination $dest -Recurse -Container -Exclude @('*.pdb','*.config')

Answer 4

As comments format code badly I'll post as answer but it's just an addition to @landyman's answer. The proposed script has a drawback - it will create double-nested folders. For example for 'd:\t1\sub1' it will create empty directory 'd:\t2\sub1\sub1'. That's due to the fact that Copy-Item for directories expects parent directory name in -Destination property not directory name itself. Here's a workaround I found:

Get-ChildItem -Path $from -Recurse -Exclude $exclude | Copy-Item -Force -Destination {
  if ($_.GetType() -eq [System.IO.FileInfo]) {
    Join-Path $to $_.FullName.Substring($from.length)
  } else {
    Join-Path $to $_.Parent.FullName.Substring($from.length)
  }
}

Answer 5

Note that the syntax spec calls for a STRING ARRAY; ala String[]

SYNTAX
    Copy-Item [[-Destination] <String>] [-Confirm] [-Container] [-Credential <PSCredential>] [-Exclude <String[]>] [-Filter <String>] [-Force] [-FromSession <PSSession>] [-Include 
    <String[]>] -LiteralPath <String[]> [-PassThru] [-Recurse] [-ToSession <PSSession>] [-UseTransaction] [-WhatIf] [<CommonParameters>]

If you're not explicit in your array generation, you end up with an Object[] - and that is ignored in many cases, leaving the appearance of "buggy behavior" because of type-safety. Since PowerShell can process script-blocks, evaluation of other than a type-specific variable (so that a valid string could be determined) would leave an opening for the potential of an injection mode attack on any system whose execution policy were lax.

So this is unreliable:

PS > $omissions = @("*.iso","*.pdf","*.zip","*.msi")
PS > $omissions.GetType()

Note the result....
IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array

And this works.... for example:

PS > $omissions = [string[]]@("*.iso","*.pdf","*.zip","*.msi")
**or**
PS > [string[]]$omissions = ("*.iso,*.pdf,*.zip,*.msi").split(',')
PS > $omissions.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     String[]                                 System.Array

Note that even a "single" element would still require the same cast, so as to create a 1-element array.

If you're trying this at home, be sure to use the Replace-Variable "omissions" to clean out the existence of $omissions before recasting it in the examples shown above.

And as far as a pipeline that works reliably that I've tested....

--------------------------------------------------------------------------------------- cd $sourcelocation

ls | ?{$_ -ne $null} | ?{$_.BaseName -notmatch "^\.$"} | %{$_.Name} | cp -Destination $targetDir -Exclude $omissions -recurse -ErrorAction silentlycontinue 
---------------------------------------------------------------------------------------

The above does a directory listing of the source files in the base (selected "current") directory, filters out potential problem items, converts the file to the basename and forces cp (copy-item alias) to re-access the file "by name" in the "current directory" - thus reacquiring the file object, and copies it. This will create empty directories, including those that may even contain excluded files (less the exclusions of course). Note also that "ls" (get-childitem) does NOT -recurse - that is left to cp. Finally - if you're having problems and need to debug, remove the -ErrorAction silentlycontinue switch and argument, which hides a lot of nuisances that might interrupt the script otherwise.

For those whose comments were related to "\" inclusions, keep in mind that you're working over the .NET sub-layer via an interpreter (i.e. PowerShell), and in c# for example, the inclusion of a single "\" (or multiple singles in a string), results in the compiler demanding you correct the condition by using either "\\" to escape the backslash, or precede the string with an @ as in @"\"; with the other remaining option being the enclosure of the string in single quotes, as '\'. All of this is because of ASCII interpolation of character combinations like "\n" etc.

The latter is a much bigger subject, so I'll leave you with that consideration.

Answer 6

I had this problem, too, and spent 20 minutes with applying the solutions here, but kept having problems.
So I chose to use robocopy - OK, it's not powershell, but should be available everywhere where powershell runs.

And it worked right out of the box:

robocopy $source $dest /S /XF <file patterns to exclude> /XD <directory patterns to exclude>

e.g.

robocopy $source $dest /S /XF *.csproj /XD obj Properties Controllers Models

Plus, it has tons of features, like resumable copy. Docs here.

Answer 7

$sourcePath="I:\MSSQL\Backup\Full"
$excludedFiles=@("MASTER", "DBA", "MODEL", "MSDB")
$sourceFiles=(ls $sourcePath -recurse -file) | where-object { $_.directory.name -notin $excludedFiles }

this is what i did, i needed to copy out a bunch of backup files to a separate location on the network for client pickup. we didn't want them to have the above system DB backups.

Answer 8

I had a similar problem extending this a bit. I want a solution working for sources like

$source = "D:\scripts\*.sql"

too. I found this solution:

function Copy-ToCreateFolder
{
    param(
        [string]$src,
        [string]$dest,
        $exclude,
        [switch]$Recurse
    )

    # The problem with Copy-Item -Rec -Exclude is that -exclude effects only top-level files
    # Copy-Item $src $dest    -Exclude $exclude       -EA silentlycontinue -Recurse:$recurse
    # http://stackoverflow.com/questions/731752/exclude-list-in-powershell-copy-item-does-not-appear-to-be-working

    if (Test-Path($src))
    {
        # Nonstandard: I create destination directories on the fly
        [void](New-Item $dest -itemtype directory -EA silentlycontinue )
        Get-ChildItem -Path $src -Force -exclude $exclude | % {

            if ($_.psIsContainer)
            {
                if ($Recurse) # Non-standard: I don't want to copy empty directories
                {
                    $sub = $_
                    $p = Split-path $sub
                    $currentfolder = Split-Path $sub -leaf
                    #Get-ChildItem $_ -rec -name  -exclude $exclude -Force | % {  "{0}    {1}" -f $p, "$currentfolder\$_" }
                    [void](New-item $dest\$currentfolder -type directory -ea silentlycontinue)
                    Get-ChildItem $_ -Recurse:$Recurse -name  -exclude $exclude -Force | % {  Copy-item $sub\$_ $dest\$currentfolder\$_ }
                }
            }
            else
            {

                #"{0}    {1}" -f (split-path $_.fullname), (split-path $_.fullname -leaf)
                Copy-Item $_ $dest
            }
        }
    }
}

Answer 9

Get-ChildItem with Join-Path was working mostly for me, but I realized it was copying root directories inside the other root directories, which was bad.

For example

• c:\SomeFolder
• c:\SomeFolder\CopyInHere
• c:\SomeFolder\CopyInHere\Thing.txt
• c:\SomeFolder\CopyInHere\SubFolder

• c:\SomeFolder\CopyInHere\SubFolder\Thin2.txt

• Source Directory: c:\SomeFolder\CopyInHere

• Destination Directory: d:\PutItInHere

Goal: Copy every childitem Inside c:\SomeFolder\CopyInHere to the root of d:\PutItInHere, but not including c:\SomeFolder\CopyInHere itself.
- E.g. Take all the children of CopyInHere and make them Children of PutItInHere

The above examples do this most of the way, but what happens is It Creates a folder Called SubFolder, and Creates a Folder in Folder called SubFolder.

That's because Join-Path Calculates a destination path of d:\PutItInHere\SubFolder for the SubFolder child item, so SubFolder get's created in a Folder called SubFolder.

I got around this by Using Get-ChildItems to bring back a collection of the items, then using a loop to go through it.

Param(
[Parameter(Mandatory=$True,Position=1)][string]$sourceDirectory,
[Parameter(Mandatory=$True,Position=2)][string]$destinationDirectory
)
$sourceDI = [System.IO.DirectoryInfo]$sourceDirectory
$destinationDI = [System.IO.DirectoryInfo]$destinationDirectory
$itemsToCopy = Get-ChildItem $sourceDirectory -Recurse -Exclude @('*.cs', 'Views\Mimicry\*')
foreach ($item in $itemsToCopy){        
    $subPath = $item.FullName.Substring($sourceDI.FullName.Length)
$destination = Join-Path $destinationDirectory $subPath
if ($item -is [System.IO.DirectoryInfo]){
    $itemDI = [System.IO.DirectoryInfo]$item
    if ($itemDI.Parent.FullName.TrimEnd("\") -eq $sourceDI.FullName.TrimEnd("\")){      
        $destination = $destinationDI.FullName  
    }
}
$itemOutput = New-Object PSObject 
$itemOutput | Add-Member -Type NoteProperty -Name Source -Value $item.FullName
$itemOutput | Add-Member -Type NoteProperty -Name Destination -Value $destination
$itemOutput | Format-List
Copy-Item -Path $item.FullName -Destination $destination -Force
}

What this does in short, is it uses the current item's full name for the destination calculation. However it then checks to see if it is a DirectoryInfo object. If it is it checks if it's Parent Folder is the Source Directory, that means the current folder being iterated is a direct child of the source directory, as such we should not append it's name to the destination directory, because we want that folder to be created in the destination directory, not in a folder of it's in the destination directory.

Following that, every other folder will work fine.

Answer 10

The exclude parameter won't work with dirs. A variant of Bo's script does the trick:

$source = 'c:\tmp\foo'
$dest = 'c:\temp\foo'
$exclude = '\.bak'
Get-ChildItem $source -Recurse  | where {$_.FullName -notmatch $exclude} | 
    Copy-Item -Destination {Join-Path $dest $_.FullName.Substring($source.length)}

Answer 11

I was looking for a way to copy files modified after a certain date/timestamp so as to archive them. This way I could save off exactly what files I worked on (assuming I know when I started). (Yes, I know this is what SCM is for, but there are times when I just want to snapshot my work without checking it in.)

Using landyman's tip, and stuff I found elsewhere, I found that this worked:

$source = 'c:\tmp\foo'
$dest = 'c:\temp\foo'
$exclude = @('*.pdb', '*.config')
Get-ChildItem $source -Recurse -Exclude $exclude |  
    where-object {$_.lastwritetime -gt "8/24/2011 10:26 pm"} | 
    Copy-Item -Destination {Join-Path $dest $_.FullName.Substring($source.length)}

Answer 12

I think the best way is to use Get-ChildItem and pipe in the Copy-Item command.

I found that this worked:

$source = 'd:\t1'
$dest = 'd:\t2'
$exclude = @('*.pdb','*.config')
Get-ChildItem $source -Recurse -Exclude $exclude | Copy-Item -Destination {Join-Path $dest $_.FullName.Substring($source.length)}

Basically, what is happening here is that you're going through the valid files one by one, then copying them to the new path. The 'Join-Path' statement at the end is so that the directories are also kept when copying over the files. That part takes the destination directory and joins it with the directory after the source path.

I got the idea from here, and then modified it a bit to make it work for this example.

I hope it works!