Reputation: 273
I'm sure this is terribly wrong, and I'm having a couple of problems. I've written out an array of WIN32_FIND_DATAW
structures to disk, one after another, and I'd like to consume and parse them in my Python script.
The code I'm currently using is:
>>> fp = open('findData', 'r').read()
>>> data = ctypes.cast(fp, ctypes.POINTER(wintypes.WIN32_FIND_DATAW))
>>> print str(data[0].cFileName)
The first problem is that the third line doesn't print a nice string like I would expect. Instead of printing $Recycle.Bin
it prints UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)
This is the result of just printing the data stored there:
>>> data[0].cFileName
u'\U00520024\U00630065\U00630079\U0065006c\U0042002e\U006e0069'
This looks relatively reasonable. $
is ASCII 0x24, R
is ASCII 0x52 and so on.
So why can't I print it like a string?
My second question is that doing:
>>> data[1].cFileName
Gives me ridiculous data. I'm fairly sure I'm not using that ctypes.cast
correctly. How should I be doing it to access these? To clarify, in C, I'd just point a PWIN32_FIND_DATAW
pointer to the beginning of the buffer and access the individual structs in the array using similar code, and I'm trying to do the same in Python.
Update
Doing:
>>> data[0].cFileName.encode('windows-1252')
Yields this error:
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-5: character maps to <undefined>
Update
The beginning of the first entry (data[0]
up to the first part of cFileName) looks like the following:
user@ubuntu:~/data$ hexdump -C findData | head -n 6
00000000 16 00 00 00 dc 5a 9f d2 31 04 ca 01 ba 81 89 1a |.....Z..1.......|
00000010 81 e2 cd 01 ba 81 89 1a 81 e2 cd 01 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 00 00 00 00 24 00 52 00 |............$.R.|
00000030 65 00 63 00 79 00 63 00 6c 00 65 00 2e 00 42 00 |e.c.y.c.l.e...B.|
00000040 69 00 6e 00 00 00 00 00 00 00 00 00 00 00 00 00 |i.n.............|
00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
I can post more data if needed.
Upvotes: 2
Views: 10848
Reputation: 32392
You should be using the struct
module from the standard library http://docs.python.org/2/library/struct.html since you are parsing a binary file format. The ctypes
module is used for integrating shared libraries (DLLs) with a binary API into a Python app. I'm not saying that what you are trying to do is not possible, but using ctypes
is more complicated that simply parsing C structs from a binary file.
Just remember that in C there is no such thing as a PWIN32_FIND_DATAW pointer. This is just a typedef that will resolve down to one of the raw C datatypes such as a 32-bit pointer, a 64-bit pointer, etc. The data in the file represents the raw base C datatypes.
In answer to comment... Avoid looking for shortcuts. You really do need deep understanding of the bits that are being written to the file and how they are organized. For that you will likely need to do some hexdumps and check the actual data representation. According to MS http://msdn.microsoft.com/en-ca/library/windows/desktop/aa365740(v=vs.85).aspx this is not a real complex structure. If the structure in wintypes doesn't work for you it is possible that you have found a bug. It is also possible that the on-disk structure is not identical to the in-ram structure. Often an in-ram data structure includes padding to maintain alignment on 16 or 64 byte boundaries. But programmers have been known to NOT dump the struct as is, but to pick it apart and output to a file minus the padding. Since ctypes/wintypes is intended for making binary api calls to a DLL its bias would be to include padding in the data layout. But the file might not include this.
Upvotes: 0
Reputation: 2079
As already mentioned in the comments, this is due to differences between windows and linux. The ctypes
module tries to fit into the local environment, hence the mismatch. The best solution is to use the struct
module to handle it in a platform independent manner. The following code shows how this can be done for a single record.
# Setup test data based on incomplete sample
bytes = "\x16\x00\x00\x00\xdc\x5a\x9f\xd2\x31\x04\xca\x01\xba\x81\x89\x1a\x81\xe2\xcd\x01\xba\x81\x89\x1a\x81\xe2\xcd\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x24\x00\x52\x00\x65\x00\x63\x00\x79\x00\x63\x00\x6c\x00\x65\x00\x2e\x00\x42\x00\x69\x00\x6e\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
bytes = bytes + "\x00"*(592-len(bytes))
import struct
import codecs
# typedef struct _WIN32_FIND_DATA {
# DWORD dwFileAttributes;
# FILETIME ftCreationTime;
# FILETIME ftLastAccessTime;
# FILETIME ftLastWriteTime;
# DWORD nFileSizeHigh;
# DWORD nFileSizeLow;
# DWORD dwReserved0;
# DWORD dwReserved1;
# TCHAR cFileName[MAX_PATH];
# TCHAR cAlternateFileName[14];
fmt = "<L3Q4L520s28s"
attrs, creation, access, write, sizeHigh, sizeLow, reserved0, reserved1, name, alternateName = struct.unpack(fmt, bytes)
name = codecs.utf_16_le_decode(name)[0].strip('\x00')
alternateName = codecs.utf_16_le_decode(alternateName)[0].strip('\x00')
print name
NOTE: This assumes that the size of MAX_PATH is 260 (which should be true, but you never know).
To read all values from the file you need to read blocks of 592 bytes at a time and then decode it as above.
Upvotes: 3