CODEWITHSUNDEEP
assemblyx86gdbendianness
samuelbrody1249
samuelbrody1249

Reputation: 4777

Flipping endianness in gdb?

In gdb I'm a bix perplexed as to why sometimes it prints bytes left-to-right and sometimes right-to-left. Here is an example where no instruction has been run in between:

>>> x/4b $rbp-4
0x7fffffffe43c: 0x04    0x00    0x03    0x16
>>> x/2h $rbp-4
0x7fffffffe43c: 0x0004  0x1603

Why is this done in gdb? I would think it would always print them as:

04 00 03 16

Upvotes: 4

Views: 5556

Answers (2)

samuelbrody1249
samuelbrody1249

Reputation: 4777

Peter has the correct and accepted answer but I'll put a very brief explanation here as well that has helped my understanding.

So let’s say we want to store the number 22 (0x16 in hex) as a two-byte value in memory.

Because x86 is little-endian (store most significant byte at highest memory address), we would store it like this:

0xAA1: 0x16                       (LSB)
0xAA2: 0x00                       (MSB)

Now, if we print it byte-by-byte in increasing order of memory address, it obviously looks like this:

0xAA1:  0x16 0x00                (2 bytes, byte by byte in increasing memory address)

But then if we want the original value of 22 back we need to interpret the two-byte value as being little-endian so it returns the correct value:

0xAA1:  0x0016 # 22 in decimal  (bytes flipped back to interpret it properly in program

Upvotes: 1

Peter Cordes
Peter Cordes

Reputation: 365852

GDB uses the target machine's native endiannes (by default1) to interpret chunks of the size you requested as integers, with addresses increasing from left to right between chunks. x86 is little-endian.

Within a chunk, 04 00 interpreted as a 16-bit little-endian integer is 0x0004. This makes GDB dump an array of short the way you'd expect, for example. You asked GDB for 16-bit integer chunks, so it does what you told it, showing the integer value using standard place-value notation with the leftmost digit the most significant. It's not trying to print the bytes separately because you asked for half-words.

If you want bytes in memory order, use b. That's what it's for.

Footnote 1:

You can change GDB's endianness setting; show endian shows the current setting. However, set endian big causes problems, corrupting register values. e.g. when stopped at _start:

(gdb) p /x $rsp
$1 = 0x7fffffffe6d0       # this is normal for x86-64
(gdb) set endian big
The target is assumed to be big endian
(gdb) x /16xw $rsp
0xd0e6ffffff7f0000:     Cannot access memory at address 0xd0e6ffffff7f0000
(gdb) p /x $rsp
$2 = 0xd0e6ffffff7f0000   # this is obviously broken, byte-reversed

Registers don't have an endianness, and flipping them when expanding their value as an address for another command is just totally broken.

Related not exact duplicates:

Upvotes: 4

Related Questions