Note for future self if .NET hash calculations from `ComputeHash()` are slower than expected
Posted by jpluimers on 2025/03/26
Normally when calculating hashes in .NET you use the [Wayback/Archive] HashAlgorithm.ComputeHash Method (System.Security.Cryptography) | Microsoft Learn.
This can be slow as [Wayback/Archive] cmcginty showed while answering the question [Wayback/Archive] How to get an MD5 checksum in PowerShell – Stack Overflow by [Wayback/Archive] Luke101 posing a faster solution (in this case for md5, but it can be generalised):
There are a lot of examples online usingComputeHash(). My testing showed this was very slow when running over a network connection. The snippet below runs much faster for me, however your mileage may vary:$md5 = [System.Security.Cryptography.MD5]::Create("MD5") $fd = [System.IO.File]::OpenRead($file) $buf = New-Object byte[] (1024*1024*8) # 8 MB buffer while (($read_len = $fd.Read($buf,0,$buf.length)) -eq $buf.length){ $total += $buf.length $md5.TransformBlock($buf,$offset,$buf.length,$buf,$offset) Write-Progress -Activity "Hashing File" ` -Status $file -percentComplete ($total/$fd.length * 100) } # Finalize the last read $md5.TransformFinalBlock($buf, 0, $read_len) $hash = $md5.Hash # Convert hash bytes to a hexadecimal formatted string $hash | foreach { $hash_txt += $_.ToString("x2") } Write-Host $hash_txt
Via my search to figure out if the Chocolatey hash check would be case-insensitive (it indeed is case-insensitive):
- [Wayback/Archive] choco/Get-CheckSumValid.ps1 at develop · chocolatey/choco
- [Wayback/Archive] choco/checksum.exe at develop · chocolatey/choco
- [Wayback/Archive] Improve handling for checksum type · Issue #1018 · chocolatey/choco
- [Wayback/Archive] chocolatey/checksum: Validates MD5/SHA1 CheckSums on the command line.
- [Wayback/Archive] checksum/Program.cs at master · chocolatey/checksum
78 //todo: Wonder if we need to flip this for perf on very large files: http://stackoverflow.com/a/13926809 79 var hash = hash_util.ComputeHash(File.Open(configuration.FilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)); 80 81 string hash_string = BitConverter.ToString(hash).Replace("-",string.Empty); 82 83 if (string.IsNullOrWhiteSpace(configuration.HashToCheck)) 84 { 85 Console.WriteLine(hash_string); 86 pause_execution_if_debug(); 87 Environment.Exit(0); 88 } 89 90 var result = string.Compare(configuration.HashToCheck, hash_string, ignoreCase: true,culture:CultureInfo.InvariantCulture); 91 92 if (result != 0) 93 { 94 Console.WriteLine("Error - hashes do not match. Actual value was '{0}'.", hash_string); 95 Environment.ExitCode = 1; 96 } 97 else 98 { 99 Console.WriteLine("Hashes match."); 100 }
--jeroen
The Wiert Corner - irregular stream of stuff 
Leave a comment