CODEWITHSUNDEEP
python
occvtech
occvtech

Reputation: 463

Python - Efficient way to flip bytes in a file?

I've got a folder full of very large files that need to be byte flipped by a power of 4. So essentially, I need to read the files as a binary, adjust the sequence of bits, and then write a new binary file with the bits adjusted.

In essence, what I'm trying to do is read a hex string hexString that looks like this: "00112233AABBCCDD"

And write a file that looks like this: "33221100DDCCBBAA"

(i.e. every two characters is a byte, and I need to flip the bytes by a power of 4)

I am very new to python and coding in general, and the way I am currently accomplishing this task is extremely inefficient. My code currently looks like this:

import binascii

with open(myFile, 'rb') as f:
        content = f.read()

hexString = str(binascii.hexlify(content))

flippedBytes = ""
inc = 0

while inc < len(hexString):
    flippedBytes += file[inc + 6:inc + 8]
    flippedBytes += file[inc + 4:inc + 6]
    flippedBytes += file[inc + 2:inc + 4]
    flippedBytes += file[inc:inc + 2]
    inc += 8

..... write the flippedBytes to file, etc

The code I pasted above accurately accomplishes what I need (note, my actual code has a few extra lines of: "hexString.replace()" to remove unnecessary hex characters - but I've left those out to make the above easier to read). My ultimate problem is that it takes EXTREMELY long to run my code with larger files. Some of my files I need to flip are almost 2gb in size, and the code was going to take almost half a day to complete one single file. I've got dozens of files I need to run this on, so that timeframe simply isn't practical.

Is there a more efficient way to flip the HEX values in a file by a power of 4?

.... for what it's worth, there is a tool called WinHEX that can do this manually, and only takes a minute max to flip the whole file.... I was just hoping to automate this with python so we didn't have to manually use WinHEX each time

Upvotes: 1

Views: 1790

Answers (2)

Jan Christoph Terasa
Jan Christoph Terasa

Reputation: 5935

You want to convert your 4-byte integers from little-endian to big-endian, or vice-versa. You can use the struct module for that:

import struct

with open(myfile, 'rb') as infile, open(myoutput, 'wb') as of:
    while True:
        d = infile.read(4)
        if not d:
            break
        le = struct.unpack('<I', d)
        be = struct.pack('>I', *le)
        of.write(be)

Upvotes: 2

Raymond Hettinger
Raymond Hettinger

Reputation: 226376

Here is a little struct awesomeness to get you started:

>>> import struct
>>> s = b'\x00\x11\x22\x33\xAA\xBB\xCC\xDD'
>>> a, b = struct.unpack('<II', s)
>>> s = struct.pack('>II', a, b)
>>> ''.join([format(x, '02x') for x in s])
'33221100ddccbbaa'

To do this at full speed for a large input, use struct.iter_unpack

Upvotes: 1

Related Questions