CODEWITHSUNDEEP
powershellsearchcopy
val
val

Reputation: 1709

Is it possible to increase the speed of this search and copy snippet?

I've got this script to search through a remote server for specific files I've listed in a CSV. However the process is very slow and I'm wondering if it's my code that makes it so. Can I improve it somehow? I've thought that simply making a directory list and searching it for the files in my CSV might be faster to avoid recursively looking through every file.

$source = "\\server1\Scanning_Maps"
$destination = "\\server1\Transfer\TEST"

$searchFiles = Import-CSV 'C:\Users\user1\Desktop\test.csv' -Header ("filename")

ForEach($File in $searchFiles) 
{
    Get-ChildItem -Path $source -Recurse | Where-Object { $_.Name -match $File.filename } | Copy-Item -Destination $destination    
}

Upvotes: 1

Views: 48

Answers (2)

Matt
Matt

Reputation: 46710

You would need to do some testing but on top of jisaak's findings you can reduce this even further by only returning files you want to copy as supposed to all files and then post processing.

I've thought that simply making a directory list and searching it for the files in my CSV might be faster to avoid recursively looking through every file.

Using -Include you can pass it the array of filenames and it should only search for and return the files that you want.

Also if you are adding the header anyway I going to guess that you have only a one column csv? If that is the case you don't even need to bother with that cmdlet and only use Get-Content. 

$searchFiles = Get-Content 'C:\Users\user1\Desktop\test.csv'

If that is a CSV file and you are using header to just work the first column then ignore that tidbit. If it is a csv we should at least expand the column so you just have a string array instead of an object one. 

$searchFiles = Import-CSV 'C:\Users\user1\Desktop\test.csv' -Header ("filename") | Select-Object -ExpandProperty filename

Now lets try using -Include. Note that it only works with -Recurse and might do nothing without it. 

Get-ChildItem -Path $source -Include $searchFiles -Recurse | Copy-Item -Destination $destination

There, no more foreach loop required. That should be even faster.

Upvotes: 2

Martin Brandl
Martin Brandl

Reputation: 58991

You run the Get-ChildItem cmdlet for every file in $searchFiles you should put it outside the foreachloop:

$source = "\\server1\Scanning_Maps"
$destination = "\\server1\Transfer\TEST"

$searchFiles = Import-CSV 'C:\Users\user1\Desktop\test.csv' -Header ("filename")

$sourceList = Get-ChildItem -Path $source -Recurse 

ForEach($File in $searchFiles) 
{
     $sourceList | Where-Object { $_.Name -match $File.filename } | Copy-Item -Destination $destination    
}

Upvotes: 1

Related Questions