endians, languages, hardwares and network

Question

We know the endian is related to the way how computers store data. Big endian computer architectures include the IBM 370, the Motorola 68000 and Sun Sparc. Little endian computers include the intel series (80486, pentium etc) and VAX.

Java is always Big-Endian because of the JVM. Network should always be Big-Endian because of the protocol.

1. C, C++ and C# depand on the computer they are running?
2. Network should always be Big-Endian because of the protocol. how about if we don't call htons and htonl before we send? The data sent across will be Little-endian if the sender is C++ on an intel machine. Is it right?
3. So we don't need to care about the endian (call ntohl and htonl), if we know all the clients and server will use computers with the same architectures and will use the same program language. is it right?

Answer 1

In very abstract terms, the one and only time when you must be endian-aware and endian-specific is when you serialize data. This has a very precise meaning which is actually covered by the language standard in C++ to some extent:

Inside the main part of your program, data comes in variables of a certain type, written T x;. So far so portable; your program always does what you want and you don't need to know how x is represented internally. You know that the memory for x starts at &x and is sizeof(T) bytes long, but you don't know anything else. If you did want to find out, you would have to cast &x from T* to unsigned char*.

While casting pointers in general is forbidden (it's called "type punning"), this particular cast is expressly permitted by the standard. Casting to char-pointer is the only way you can serialize your data from an opaque type T into a stream of actual bytes. It is precisely at this moment that you must know about endianness (or more generally, representation), because you must know in which order the byte stream makes up the internal representation of T.

For integral types you can do without casting pointers, but the interface is still at the conversion from byte stream to value:

unsigned char buf[sizeof(unsigned int)];
unsigned int value;

buf[0] = value; buf[1] = value >> 8; buf[2] = value >> 16; /*...*/  // We chose an endianness!
value = buf[0] + (buf[1] << 8) + (buf[2] << 16) + ... ; // ditto

You will find the need to convert values into bytestreams and vice versa when using operations like read and write, usually associated to files, streams or sockets.

Note that for integral values we never need to know about the endianness of the program itself - we only need to know the endianness that is used by the byte stream!

Answer 2

1. For C and C++, at least, yes; the endianness typically depends on the machine (but may also depend on the compiler). For C#, I don't know.
2. Many network protocols are big-endian, yes. If you don't call htonl, then you will not be creating a valid packet on a little-endian machine.
3. So you should always call htonl, etc. (or the equivalent in whichever language you're using). Because if even if you have a homogeneous environment today, it's almost certain that in the future, this will change.

More specifically, you should always do the conversion as close to the interface as you can, and in one place. If you have endianness conversion calls strewn across your codebase, it becomes difficult to reason about whether your code is sane or not.

Answer 3

Strictly Java uses the same endian as the hardware it is running on, but it does not show for the JVM user as you cannot access raw memory in Java.

1. Right, C languages uses the layout that the currently running processor uses.
2. Correct.
3. It is good practice to always convert to network byte order regardless. Sooner or later you are going to regret that you did not use htons (and others) just because for the time being it did not matter. The cost is normally minimal, so do it unless you have a very good reason not to!

Answer 4

1. Data transfered between computers in binary depends on Endian ordering.

2. C, C++ and C# do not make any demands or requirements on Endianess.

3. Network should follow protocol. The numbers are converted to internal format after they are input and written out per protocol. They can be any format for internal processing.

4. Only worry about Endianess when transferring binary data between computers, whether stored in files or immediately transferred.

5. Floating point numbers suffer from similar problems.

6. Many languages do not care about Endianness.