Different SHA1 hash result in Powershell and Ruby

Question

We have script written in powershell to get some information from users terminal. And have created SHA1 of that information, just to verify integrity of result.

On server side where we get these information from multiple terminals, we are verifying each results and this side of script is being written in Ruby.

Now problem, is that both these script are giving different SHA1 hash, and I am not sure where I am making mistake.

Powershell code is as given below.

$String = "

    Directory: D:\OneDrive -
    ControlCase\jt-work\evidance-collection\evidances-text\PCI_Evidences_CCIN-CAS-VKAUS\evidences


Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---        2019-01-16   1:14 PM       7073 21_to_calc_hash.ps1
-a---        2019-01-16   1:15 PM       9973 CCIN-CAS-VKAUS_pci_evidence_Q21.txt
-a---        2019-01-15   9:37 PM      67399 CCIN-CAS-VKAUS_pci_evidence_Q23.txt
-a---        2019-01-15   9:37 PM       5055 CCIN-CAS-VKAUS_pci_evidence_Q34.txt
-a---        2019-01-15   9:38 PM      10820 CCIN-CAS-VKAUS_pci_evidence_Q45.txt
-a---        2019-01-15   9:38 PM      13129 CCIN-CAS-VKAUS_pci_evidence_Q50.txt
-a---        2019-01-15   9:38 PM       7163 CCIN-CAS-VKAUS_pci_evidence_Q67.txt
-a---        2019-01-15   9:39 PM       4301 CCIN-CAS-VKAUS_pci_evidence_Q69.txt
-a---        2019-01-15   9:39 PM       2900 CCIN-CAS-VKAUS_pci_evidence_Q81.txt

"

Function Get-Hash([String] $String)
{ 
$StringBuilder = New-Object System.Text.StringBuilder 
[System.Security.Cryptography.HashAlgorithm]::Create("sha1").ComputeHash([System.Text.Encoding]::UTF8.GetBytes($String))|%{ 
[Void]$StringBuilder.Append($_.ToString("x2")) 
} 
$StringBuilder.ToString() 
}

Get-Hash($String)

and this the hash we get for above script e7c527068445c52635287b9ecf55566c2564d595

below is Ruby script

require 'digest'
varj = "

Directory: D:\OneDrive -
ControlCase\jt-work\evidance-collection\evidances-text\PCI_Evidences_CCIN-CAS-VKAUS\evidences


Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---        2019-01-16   1:14 PM       7073 21_to_calc_hash.ps1
-a---        2019-01-16   1:15 PM       9973 CCIN-CAS-VKAUS_pci_evidence_Q21.txt
-a---        2019-01-15   9:37 PM      67399 CCIN-CAS-VKAUS_pci_evidence_Q23.txt
-a---        2019-01-15   9:37 PM       5055 CCIN-CAS-VKAUS_pci_evidence_Q34.txt
-a---        2019-01-15   9:38 PM      10820 CCIN-CAS-VKAUS_pci_evidence_Q45.txt
-a---        2019-01-15   9:38 PM      13129 CCIN-CAS-VKAUS_pci_evidence_Q50.txt
-a---        2019-01-15   9:38 PM       7163 CCIN-CAS-VKAUS_pci_evidence_Q67.txt
-a---        2019-01-15   9:39 PM       4301 CCIN-CAS-VKAUS_pci_evidence_Q69.txt
-a---        2019-01-15   9:39 PM       2900 CCIN-CAS-VKAUS_pci_evidence_Q81.txt

"

varj_Encoded = varj.encode(Encoding::ISO_8859_1)
puts Digest::SHA1.hexdigest(varj_Encoded)

and here is the SHA1 has we get from this script. 77f7cecb2b75c16b1e929a56b644fad7d7f95965

Now condition here is, I can't make/propose any changes in Powershell part. I need to tune ruby code to make it match.

Answer 1

The encoding makes no difference in this situation, as we're only dealing with ASCII characters, and they both encode ASCII the same way. The problem is

1. Your data is different in both cases, note the spaces preceding Directory and ControlCase in your first example but not present in the second.

2. You need to escape the backslashes in the ruby string, or it interprets them as escape characters

Once you resolve these two issues, you will get the same result:

PS:

PS H:\> $String = "
>>
>>     Directory: D:\OneDrive -
>>     ControlCase\jt-work\evidance-collection\evidances-text\PCI_Evidences_CCIN-CAS-VKAUS\evidences
>>
>>
>> Mode                LastWriteTime     Length Name
>> ----                -------------     ------ ----
>> -a---        2019-01-16   1:14 PM       7073 21_to_calc_hash.ps1
>> -a---        2019-01-16   1:15 PM       9973 CCIN-CAS-VKAUS_pci_evidence_Q21.txt
>> -a---        2019-01-15   9:37 PM      67399 CCIN-CAS-VKAUS_pci_evidence_Q23.txt
>> -a---        2019-01-15   9:37 PM       5055 CCIN-CAS-VKAUS_pci_evidence_Q34.txt
>> -a---        2019-01-15   9:38 PM      10820 CCIN-CAS-VKAUS_pci_evidence_Q45.txt
>> -a---        2019-01-15   9:38 PM      13129 CCIN-CAS-VKAUS_pci_evidence_Q50.txt
>> -a---        2019-01-15   9:38 PM       7163 CCIN-CAS-VKAUS_pci_evidence_Q67.txt
>> -a---        2019-01-15   9:39 PM       4301 CCIN-CAS-VKAUS_pci_evidence_Q69.txt
>> -a---        2019-01-15   9:39 PM       2900 CCIN-CAS-VKAUS_pci_evidence_Q81.txt
>>
>> "
PS H:\> Get-Hash($string)
6454c0ecf1700448fb2496037a1e9ce496b185cd

Ruby:

>> varj = " 
..  
..     Directory: D:\\OneDrive - 
..     ControlCase\\jt-work\\evidance-collection\\evidances-text\\PCI_Evidences_CCIN-CAS-VKAUS\\evidences 
..  
..  
.. Mode                LastWriteTime     Length Name 
.. ----                -------------     ------ ---- 
.. -a---        2019-01-16   1:14 PM       7073 21_to_calc_hash.ps1 
.. -a---        2019-01-16   1:15 PM       9973 CCIN-CAS-VKAUS_pci_evidence_Q21.txt 
.. -a---        2019-01-15   9:37 PM      67399 CCIN-CAS-VKAUS_pci_evidence_Q23.txt 
.. -a---        2019-01-15   9:37 PM       5055 CCIN-CAS-VKAUS_pci_evidence_Q34.txt 
.. -a---        2019-01-15   9:38 PM      10820 CCIN-CAS-VKAUS_pci_evidence_Q45.txt 
.. -a---        2019-01-15   9:38 PM      13129 CCIN-CAS-VKAUS_pci_evidence_Q50.txt 
.. -a---        2019-01-15   9:38 PM       7163 CCIN-CAS-VKAUS_pci_evidence_Q67.txt 
.. -a---        2019-01-15   9:39 PM       4301 CCIN-CAS-VKAUS_pci_evidence_Q69.txt 
.. -a---        2019-01-15   9:39 PM       2900 CCIN-CAS-VKAUS_pci_evidence_Q81.txt 
..  
.. "
>> puts Digest::SHA1.hexdigest(varj.encode(Encoding::UTF_8))
=> 6454c0ecf1700448fb2496037a1e9ce496b185cd
>> puts Digest::SHA1.hexdigest(varj.encode(Encoding::ISO_8859_1))
=> 6454c0ecf1700448fb2496037a1e9ce496b185cd

Edit:

If you're still not able to match, I think the best approach is to compare the byte values of each string to identify differences.

PS:

PS H:\> $enc = [system.Text.Encoding]::UTF8
PS H:\> $enc.GetBytes($String)
10
10
32
32
...

Ruby:

>> varj_Encoded.bytes.to_a
=> [10, 10, 32, 32, ...

Answer 2

As Persistent13 mentioned in a comment, the encoding difference is the most likely reason. The encoding class in .Net has a few static members corresponding to common encodings but ISO-8859-1 isn't one of them. You can see the rest with [System.Text.Encoding]::GetEncodings() and you can get a specific one with [System.Text.Encoding]::GetEncoding() (there are a few overloads).

So for example, try this:

$bytes = [System.Text.Encoding]::GetEncoding('iso-8859-1').GetBytes($string)
[System.Security.Cryptography.HashAlgorithm]::Create("sha1").ComputeHash($bytes)

Since you can't change the PowerShell part, you can do this just to test the theory in the console.

Or go right to the part of changing Ruby to use UTF8.

varj_Encoded = varj.encode(Encoding::UTF-8)
puts Digest::SHA1.hexdigest(varj_Encoded)