CODEWITHSUNDEEP
c#performancehashsettings
jasonh
jasonh

Reputation: 30303

Quickly determining if a set of files are identical in C#?

I have the need to relatively quickly be able to determine if a set of files on a user's machine has been processed before by my application. The app in question uploads the files for the user to a server, and if the files have been uploaded before, it skips the upload. My plan so far has been to hash the files and then store the results along with an identifier of how they were uploaded to the server. The problem I think I'm going to run into is that storing this data could become quite cumbersome due to the length of the hashes. I'm expecting around 30-40 files right now but that could double or (hypothetically) even triple.

Would this be possible to store using a Dictionary, with the hashes as the key and the server information as the value? I would then have that Dictionary stored in the App's Properties.Settings.Default object. Is that feasible to store with that system or will I run into some sort of problem there? Note that due to the nature of the application, there is no chance of two users ever having the same set of data, so I don't need to compare uploads between users. Additionally, what would the performance be like for this type of operation? Users are expected to have at least a Pentium-M 1.5 GHz processor with 1 GB of RAM.

Upvotes: 1

Views: 205

Answers (3)

Vivek
Vivek

Reputation: 16508

Why not compare the File Modified DateTime instead? For this you need to save the modified date on the server.

Upvotes: 0

hugoware
hugoware

Reputation: 36397

In reference to getting the hash values, I thought I'd mention this...

Using a hash value is good, so long as you get the same result each time without fail. I've read somewhere that .GetHashCode() isn't the same between different versions of .NET, so if you're planning on saving the hash in a persistent state, I'd avoid .GetHashCode(). If it is all done at once, then .GetHashCode() is ideal for comparing if things are the same.

If you need to persist the hash there are hashing classes available in .NET. I'm admittedly not an expert with this, but I think SHA1 has a hashing method.

Upvotes: 1

Jon Skeet
Jon Skeet

Reputation: 1500515

I probably wouldn't put the dictionary into the app.config file, although I guess you could, depending on the server information. I'd probably just put it in a text file on its own unless you found that to be more of a problem for some reason. It feels like it's more data for the application than configuration of the application.

Performance shouldn't be an issue at all - dictionaries are designed to still be efficient with millions of entries, let alone the tens or hundreds you're talking about.

Upvotes: 2

Related Questions