Manipulating binary C struct data off line

Question

I have a practical problem to solve. In an embedded system I'm working on, I have a pretty large structure holding system parameters, which includes int, float, and other structures. The structure can be saved to external storage.

I'm thinking of writing a PC software to change certain values inside the binary structure without changing the overall layout of the file. The idea is that one can change 1 or 2 parameter and load it back into memory from external storage. So it's a special purpose hex editor.

The way I figures it: if I can construct a table that contains: - parameter name - offset of the parameter in memory - type I should be able to change anything I want.

My problem is that it doesn't look too easy to figure out offset of each parameter programmatically. One can always manually to printf their addresses but I'd like to avoid that.

Anybody know a tool that can be of help?

EDIT The system in question is 32 ARM based running in little endian mode. Compiler isn't asked to pack the structure so for my purpose it's the same as a x86 PC. The problem is the size of the structure: it contains no less than 1000 parameters in total, spreading across multiple level of nested structures. So it's not feasible (or rather i'm not willing to) to write a short piece of code to dump all the offsets. I guess the key problem is parsing the C headers and automatically generate offsets or code to dump their offsets. Please suggest such a parsing tool.

Thanks

Answer 1

I suspect you could parse the output of the pahole tool to get this. It reads the debugging information in an executable and prints out the type, name and offset of each struct member.

Answer 2

To make sure I understand the question:

In your embedded system, you have a struct that contains system settings that looks something like this:

struct SubStruct{
    int               subSetting1;
    float             subSetting2;
}

/* ... some other substructure definitions. ... */

struct Settings{
    float             setting1;
    int               setting2;
    struct SubStruct  setting3; 
    /* ... lots of other settings ... */
}

And your problem is you want to take a binary file containing said structure and modify the values on another machine?

Given that you know the sizes of the integral types and the packing used by the embedded system, you may have two options. If the endianness of your embedded system and PC system is different, you will also have to deal with that, e.g. using hton when writing fields.

Option 1: Declare the same structure in your PC program, with pragmas to force the same byte-alignment as the embedded system, and with any integral types that are different sizes between the embedded and PC systems "retyped" to specific sizes, e.g. by using int16_t, int32_t if you have stdint.h, or your own typedefs if not, on your PC program.

Option 2: You may be able to simply make a list of types, e.g. a list that looks like:

Name                        Type
setting1                    float
setting2                    int
subStruct1SubSetting1       int
subStruct1SubSetting2       float

And then calculate the locations based on the sizes and packing used by the embedded system, e.g. if the embedded system uses 4-byte alignment, 4-byte floats, and 2-byte ints, the above table would calculate out:

Name                        Type         Calculated Offset
setting1                    float        0
setting2                    int          4
subStruct1SubSetting1       int          8
subStruct1SubSetting2       float        12

or if the embedded system packs the structure to 1-byte alignment (e.g. if it's an 8-bit micro)

Name                        Type         Calculated Offset
setting1                    float        0
setting2                    int          4
subStruct1SubSetting1       int          6
subStruct1SubSetting2       float        8

Answer 3

The way this is normally done is by just sharing the header file that defines the struct with the other (PC) application as well. You should then be able to basically load the bytes and cast them to that kind of struct and edit away.

Before doing this, you need to know about (and possibly handle):

1. Type sizes. If the two platforms are very different (16bit vs 32 bit) etc, you might have differently sized ints/floats, etc.
2. Field alignment. If the compiler/target are different, the packing rules for the fields won't be necessarily the same. You can generally force these to be the same with compiler-specific #pragmas.
3. Endianness. This is something to be careful of in all cases. If your embedded device is a different endianness from the PC target, you'll have to byte swap each field when you load on the PC.

If there are too many differences here to bother with, and you're building something quicker/dirtier, I don't know of tools to generate struct offsets automatically, but as another poster says, you could write some short C code that uses offsetof or equivalent (or does, as you suggest, pointer arithmetic) to dump out a table of fields and offsets.

Answer 4

Rather than writing your own tool, there are existing hex editors that'll take a struct definition and let you edit the values in a file. If you're using Windows, I believe that Hex Workshop can do this, for example.

Answer 5

I'm unsure what debugging capabilities you have in your system. Under Windows I would use simple debugging script with text conversion on top of it (debugger can be run in batch mode to generate type information).

Answer 6

Given a fully-defined structure in your code, you can use the offsetof() macro to find out how the fields are laid out. This won't help if you're trying to programmatically determine the layout on the other machine, though.