Different md5 sum from same file

Question

I'm counting md5 sum from video file from iphone gallery. Each time i choose the same file it has different md5 sum. I also check the data length in bytes and it stay the same. So my question is - why? Here is some code with one from many ways i was trying to make it.

- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary *)info
{   
    NSString *mediaType = [info objectForKey:UIImagePickerControllerMediaType];
    if ([mediaType isEqualToString:@"public.movie"])
    {
        NSURL *videoURL = [info objectForKey:UIImagePickerControllerMediaURL];
        videoData = [NSData dataWithContentsOfURL:videoURL];
        [videoData retain];
        NSLog(@"VIDEO DATA MD5: %@", [videoData md5]);
        NSLog(@"VIDEO DATA LEN: %d", videoData.length);
    }

    [self dismissModalViewControllerAnimated:YES];
}

Implementation of MD5 Method:

#import <CommonCrypto/CommonDigest.h>

@implementation NSData(MD5)

- (NSString*)MD5
{
  // Create byte array of unsigned chars
  unsigned char md5Buffer[CC_MD5_DIGEST_LENGTH];

  // Create 16 byte MD5 hash value, store in buffer
  CC_MD5(self.bytes, self.length, md5Buffer);

  // Convert unsigned char buffer to NSString of hex values
  NSMutableString *output = [NSMutableString stringWithCapacity:CC_MD5_DIGEST_LENGTH * 2];
  for(int i = 0; i < CC_MD5_DIGEST_LENGTH; i++) 
    [output appendFormat:@"%02x",md5Buffer[i]];

  return output;
}

@end

Answer 1

I can confirm this behavior. It happens regardless of the hash function chosen (I tried MD5, SHA1, and SHA256).

The issue appears to be NSData. I create two NSData instances from the exact same file and then use [data1 isEqualTo:data2] and that returns true. However, running those through the CC_SHA1 hash algorithm returns a different hash for each.

To work around this, drop NSData. Instead, open the file using lower-level C APIs. Here's an example in C. Note the "create" convention—YOU are responsible for releasing the returned char* buffer when you're done. Using this approach will produce an identical hash each time.

char* _Nullable createBufferWithContentsOfFileAndReportLength(NSString* _Nonnull fileToScan, long* _Nullable length)
{
    char const *pathToFile = [fileToScan cStringUsingEncoding:NSUTF8StringEncoding];
    
    FILE *file = fopen(pathToFile, "r");
    if (file == NULL) {
        NSLog(@"Unable to open file: %@", fileToScan);
        return NULL;
    }
    
    //
    //  Get the file length.
    //  'fileLength' will be the number of bytes in the file. It will not include a null terminator or EOF. Some UTF8 characters require more than one byte, so we
    //  can't guarantee that fileLength is also the number of characters.
    //  NOTE: We don't use the fseek()/ftell()/rewind approach because it's not secure. See CERT FIO19-C advisory for details.
    //
    int fd = fileno(file);
    if (fd < 0) {
        NSLog(@"Unable to get a file descriptor for: %@", fileToScan);
        fclose(file);
        return NULL;
    }
    struct stat statBuffer;
    if (fstat(fd, &statBuffer) == -1) {
        NSLog(@"Unable to retrieve stats about this file: %@", fileToScan);
        fclose(file);
        return NULL;
    }
    long fileLength = statBuffer.st_size;
    
    char *buffer = malloc(sizeof(char) * (fileLength + 1));     // enough memory to read the entire file, plus a spot for the null terminator
    if (buffer == NULL) {
        NSLog(@"Unable to allocate memory for: %@", fileToScan);
        fclose(file);
        return NULL;
    }
    
    fread(buffer, fileLength, sizeof(char), file);
    buffer[fileLength] = '\0';                              // No +1; this is zero-indexed. If fileLength is 5 characters, the 5th slot in the array needs to be the null terminator.
    fclose(file);

    if (length != NULL) {
        *length = fileLength + 1;
    }
    
    return buffer;
}