CODEWITHSUNDEEP
powershellsynchronizationcopycompareobject
siboney
siboney

Reputation: 56

Copy new files from external HD to Desktop with Powershell

Thank you for reading my post and your advice :) The scenario is, that I have an HD built in my desktop where I collected files for many years. So one day I did a backup to an external HD, which I then took travelling and kept on collecting photos from my phone etc.

Since then I changed the folder structure on my desktop a lot, so I can't compare the folders/files 1on1.

The goal is obviously, that all new files that are on my external HD get copied to my internal HD, all put in a folder named like '2SORT'.

I found a faster version of compare-object (see comments) but yet the results are not correct, it will copy a lot of files that already exist.

Here's what I got in Powershell:

cls

$path_desktop = 'C:\Files\Media\Bilder'
$path_external = 'E:\Bilder'
$path_destination = 'C:\Files\Media\Bilder\2SORT'

$ref = Get-ChildItem -path $path_desktop -Recurse -File
$com = Get-ChildItem -path $path_external -Recurse -File

$file_Diffs = Compare-Object -ReferenceObject $ref -DifferenceObject $com | Where { $_.SideIndicator -eq '=>' }

$file_Diffs | 
foreach {

	$copyParams = @{}
	$copyParams.Path = $_.InputObject.FullName
	$copyParams.Destination = "$path_destination\" + $_.InputObject.Name

	copy-Item @copyParams -force -PassThru | Write-Host
     
}

Upvotes: 0

Views: 86

Answers (2)

siboney
siboney

Reputation: 56

In the end I had to compare the files by their MD5-Hash because I got many duplicates with just different 'lastwritetime':

cls

    $path_desktop = 'C:\Files\Media\Bilder\'
    $path_external = 'C:\Files\Media\ext'
    $path_destination = 'C:\Files\Media\Bilder\2SORT'
    $path_intcsv = 'C:\Files\Media\Bilder\inthash2.csv'
    $path_extcsv = 'C:\Files\Media\Bilder\exthash2.csv'

    $count=0
    $res = @()

    Get-ChildItem -path $path_desktop -Recurse -File | % {
    	
    	Write-Host ($count++)
    	$hash = Get-FileHash $_.FullName -Algorithm MD5
    	$res += New-Object PSObject -Property @{Hash=$hash.Hash; FullName=$_.FullName}
    }

    $res | epcsv -path $path_intcsv -NoTypeInformation

    $count=0
    $res = @()

    Get-ChildItem -path $path_external -Recurse -File | % {
    	Write-Host ($count++)
    	$hash = Get-FileHash $_.FullName -Algorithm MD5
    	$res += New-Object PSObject -Property @{Hash=$hash.Hash; FullName=$_.FullName}
    }

    $res | epcsv -path $path_extcsv -NoTypeInformation
    	

    $int = Import-Csv $path_intcsv 
    $ext = Import-Csv $path_extcsv
      
    $IntHash = @{}          
    $ExtHash = @{}          
    $DifHash = @{}          
     
    $int | ForEach-Object {
    	$newKey=$_.Hash
    	if (!$IntHash.ContainsKey($newKey)) {
    		$IntHash.Add($newKey, $null)
    	}
    }               

    $ext | ForEach-Object {
    	$newKey=$_.Hash
    	if (!$ExtHash.ContainsKey($newKey)) {
    		$ExtHash.Add($newKey, $_.FullName)
    	}
    }  

    ForEach ($key in $ExtHash.Keys) {            
    	If (!$IntHash.ContainsKey($key)) {            
    		$DifHash.Add($key, $ExtHash[$key])
    	}    
    }     

    #check folder exists
    if (-not (test-path $path_destination)) { 
    		$make = new-item -ItemType directory -Path $path_destination
    }      

    #copy files
    ForEach ($key in $DifHash.Keys) {
    	$fullname = $DifHash[$key]
    	Copy-Item $fullname -Destination $path_destination -Force
    }

Upvotes: 0

Brandon McClure
Brandon McClure

Reputation: 1400

If this is something you would do regularly, I would recommend refactoring to use robocopy. After the first run any subsequent runs will be much faster because robocopy was designed to perform these tasks. The first run is going to be slow no matter what tool you use.

Compare-Object is going to compare property values. By passing a bunch of values in it will have more work to do. Also, you are not actually moving any files while this function is running, so if this operation is canceled, your script has not moved anything. If you just want to compare a list of file names in each and copy if they exist in $ref but not $com, You could limit the properties you are getting, and iterate/move over each in $ref manually like I do below.

If you want to keep using Compare-Object take a look at it's -Property parameter to limit the number of properties it is comparing.

$path_desktop = 'C:\Files\Media\Bilder'
$path_external = 'E:\Bilder'
$path_destination = 'C:\Files\Media\Bilder\2SORT'

$ref = Get-ChildItem -path $path_desktop -Recurse -File | select name, fullname
$com = Get-ChildItem -path $path_external -Recurse -File | select name, fullname

$ref | 
foreach {
    if ($_.name -in $com | select -ExpandProperty name)
        continue;

    --Copy file
}

You could also get other file properties as needed, then you could use those properties when you are doing the comparison (either "manually" or using compare-object and iterating over the diff) 

Get-ChildItem -path $path_desktop -Recurse -File | select Name,fullname, LastWriteTime

to see what properties Get-ChildItem returns, pass it into `Get-Member -Type Properties

Get-ChildItem -path $path_desktop -Recurse -File | Get-Member -Type Properties

Upvotes: 1

Related Questions