CODEWITHSUNDEEP
powershellrobocopy
HandsUp
HandsUp

Reputation: 13

File size total calculation issue using Copy-WithProgress Powershell function

I adopted a PowerShell script Copy-WithProgress written by Trevor Sullivan (thank you!) but some of the calculations seem to be off where over 8,000 files return 3PB and 5,000 files return 7Bytes.

I am using the following Robocopy parameters for my "staging log":

$CommonRobocopyParams = '/MIR /S /NP /NDL /NC /BYTES /NJH /NJS /R:1 /W:1'

And am using the following to read the log to calculate bytes to be copied:

$BackupLog = 'C:\Users\username\Desktop\Backup_2018-09-20-112849'
$StagingLogPath = '{0}\robocopy staging.log' -f $BackupLog
$StagingContent = Get-Content -Path $StagingLogPath | ? {$_} #Skip blank lines in log
$TotalFileCount = $StagingContent.Count                    

Write-Host "Number of selected files to be copied: $TotalFileCount."

$RegexBytes = '(?<=\s+)\d+(?=\s+)' 

[RegEx]::Matches(($StagingContent -join "`n"), $RegexBytes) | % { $BytesTotal = 0 } { $BytesTotal += $_.Value }

$message = 'Total number of bytes to be copied: {0:N0}' -f ($BytesTotal/1GB)
Write-Host $message

The "Skip blank lines in log" above is to handle another set of logs from another folder i.e. Documents within the same "staging log" file separated by a blank line.

The script calculates incorrectly in 1 out of 6 machines I've tested, and only when backing up the Desktop on the problem machine. I've checked the "staging log" and compared it to the correct calculation ones which looks like this:

13162   C:\Users\username\Desktop\File 1.xlsx           
1765924 C:\Users\username\Desktop\File 2.xlsx           
68838   C:\Users\username\Desktop\File 3.pdf            
2380    C:\Users\username\Desktop\File 4.docx           
403759  C:\Users\username\Desktop\File 5.txt            
10068   C:\Users\username\Desktop\File 6.xlsx           
28502   C:\Users\username\Desktop\File 7.jpg            
139277  C:\Users\username\Desktop\File 8.docx           
553469  C:\Users\username\Desktop\File 9.log            
283502  C:\Users\username\Desktop\Book1.xlsx

I'm curious if anyone has run into this issue and solved the issue. Thank you in advance!

Upvotes: 1

Views: 605

Answers (1)

Matt
Matt

Reputation: 46720

Pretty sure the pitfall here is your using System.Int32. For really large numbers that math would fail.

Consider something that supports larger numbers like [int64],[long] or [double]. For example (1pb).GetType().FullName nets System.Int64

% { [int64]$BytesTotal = 0 }

That single cast at initialization should be enough to fix any miscalculations down the line.

For craps and laughter you can look at my Copy-WithRobocopyProgress function on GitHub if you were so inclined. Uses jobs and robocopy to display progress in PowerShell

Upvotes: 1

Related Questions