CODEWITHSUNDEEP
cgccarmshared-libraries
maverik
maverik

Reputation: 5586

"undefined behaviour" on ARM device

I'm using libwebsockets library (pure C) to write an app (C++11), that is runnig on ARMv7 device. I'm using gcc 4.7.3 (arm, gnueabi) and openwrt to build the toolchain and applications.

So, libwebsockets library during the connection to the server sends the handshake request inside of HTTP request. It looks like this:

GET / HTTP/1.1
Pragma: no-cache
Cache-Control: no-cache
Host: 192.168.1.111
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: YDtqt/+y5Efpzo1YiCg5YQ==
Origin: /
Sec-WebSocket-Protocol: myproto
Sec-WebSocket-Extensions: deflate-fram
Sec-WebSocket-Version: 13

This part work well when I build my app under x86_64 linux (Fedora, gcc 4.8.1). But if I build the app for ARM and then run it, the HTTP request looks like this:

gPra ma: / HTTP/1.1
no-<ach
C chegCon rol
 no<cac�e
Host: 192.168.1.111
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: QKnxtiEc3IlvyOW254h6kg==
Origin: /
Sec-WebSocket-Protocol: myproto
Sec-WebSocket-Extensions: deflate-frame
Sec-WebSocket-Version: 13

or this (it changes every time):

GET / HTTP/1.1
Pragma: no-cache
Cache-Control: no-cache
tUpgoade2.168.1.111
we.soc

The app's code is exactly the same as for x86_64 linux box, so I guess the problem is somewhere in the library or in the toolchain (compiler, glibc). Here's the library code that makes an HTTP request (from client.c):

char *
libwebsockets_generate_client_handshake(struct libwebsocket_context *context,
    struct libwebsocket *wsi, char *pkt)
{
    char buf[128];
    char hash[20];
    char key_b64[40];
    char *p = pkt;
    int n;

    n = libwebsockets_get_random(context, hash, 16);
    if (n != 16) {
        lwsl_err("Unable to read from random dev %s\n",
                    SYSTEM_RANDOM_FILEPATH);
        libwebsocket_close_and_free_session(context, wsi,
                     LWS_CLOSE_STATUS_NOSTATUS);
        return NULL;
    }

    lws_b64_encode_string(hash, 16, key_b64, sizeof(key_b64));

    p += sprintf(p, "GET %s HTTP/1.1\x0d\x0a",
            lws_hdr_simple_ptr(wsi, _WSI_TOKEN_CLIENT_URI));

    p += sprintf(p, "Pragma: no-cache\x0d\x0a""Cache-Control: no-cache\x0d\x0a");
    p += sprintf(p, "Host: %s\x0d\x0a", lws_hdr_simple_ptr(wsi, _WSI_TOKEN_CLIENT_HOST));
    p += sprintf(p, "Upgrade: websocket\x0d\x0a""Connection: Upgrade\x0d\x0a""Sec-WebSocket-Key: ");
    strcpy(p, key_b64);
    p += strlen(key_b64);
    p += sprintf(p, "\x0d\x0a");
    if (lws_hdr_simple_ptr(wsi, _WSI_TOKEN_CLIENT_ORIGIN))
        p += sprintf(p, "Origin: %s\x0d\x0a", lws_hdr_simple_ptr(wsi, _WSI_TOKEN_CLIENT_ORIGIN));

    if (lws_hdr_simple_ptr(wsi, _WSI_TOKEN_CLIENT_SENT_PROTOCOLS))
        p += sprintf(p, "Sec-WebSocket-Protocol: %s\x0d\x0a", lws_hdr_simple_ptr(wsi, _WSI_TOKEN_CLIENT_SENT_PROTOCOLS));

    p += sprintf(p, "Sec-WebSocket-Extensions: ");
    p += sprintf(p, "\x0d\x0a");

    if (wsi->ietf_spec_revision)
        p += sprintf(p, "Sec-WebSocket-Version: %d\x0d\x0a",
                       wsi->ietf_spec_revision);

    // here my callback is called and I print out the header (see examples above) 
    context->protocols[0].callback(context, wsi,
        LWS_CALLBACK_CLIENT_APPEND_HANDSHAKE_HEADER,
        NULL, &p, (pkt + sizeof(context->service_buffer)) - p - 12);

    p += sprintf(p, "\x0d\x0a");
    return p;
}

I've removed some code and comments for brevity. Well, I'm stuck here. Can't find something that could break the whole thing. Does anyone have any ideas on what's going on? May be the fresh pair of eyes can help me.

Thanks.

Upvotes: 1

Views: 418

Answers (2)

tazmanos
tazmanos

Reputation: 313

I have just finished a long debugging session with the exact same problem (connection request corruption for libwebsockets on ARMv6), so this may save someone else from 2 days of debugging (including learning how to use GDB).

Three words: Unaligned memory access

More than three words: By mistake, my makefile did not pass the right compiler flags to gcc (especially -mno-unaligned-access) and ARMv6 (or anything below ARM11) will simply damage the 1-3 memory bytes before the misaligned write address.

EDIT: Apparently I misread the documentation and ARMv6 (and above) is supposed to be ok with unaligned access,yet mine isn't, so YMMV...

Upvotes: 2

Tom Tanner
Tom Tanner

Reputation: 9354

It looks a little like on the ARM build your pkt buffer may not be large enough or your stack isn't large enough.

One thing that worries me though is this huge buff that is declared and not used. Is it really not used or is it required by some of the code you've removed for clarity? I have to say when I've seen that sort of thing and investigated it it's being used to prevent some form of undefined behaviour caused by overflowing a buffer.

This code is littered with unchecked sprintfs and it just assumes the output buffer is big enough for the data and is pretty much a breeding ground for strange errors like you are seeing.

Upvotes: 4

Related Questions