How can tcp socket client in python talk with uwsgi?

Question

How can tcp socket client in python talk with uwsgi ?

my architectures are

tcp socket client -> nginx -> uwsgi

tcp socket client -> uwsgi

Answer 1

It's not clear what exactly you want to know, so let me start with the following question:

How can a client talk with uwsgi?

$ python -m venv venv
$ venv/bin/pip install uwsgi-tools
$ venv/bin/uwsgi_curl /var/run/uwsgi-socket example.com

(Here I install the uwsgi-tools package (GitHub) in a virtual environment, and run a uwsgi client (uwsgi_curl), passing it the path to the uwsgi server socket (/var/run/uwsgi-socket), and an URL (example.com).)

Let's us take a look at what exactly the client and the server send each other. For that let's place socat between them:

$ socat -r /tmp/request -R /tmp/response \
    UNIX-LISTEN:/tmp/proxy UNIX-CONNECT:/var/run/uwsgi-socket

(Here socat connects a Unix domain socket at /tmp/proxy with uwsgi's socket at /var/run/uwsgi-socket. The requests are stored in /tmp/request, the responses in /tmp/response. Alternatively you can use socat -v -x UNIX-LISTEN:... UNIX-CONNECT:... 2> /tmp/traffic, see -v and -x.)

And make a request:

$ python -m venv /tmp/venv
$ /tmp/venv/bin/pip install uwsgi-tools
$ /tmp/venv/bin/uwsgi_curl /tmp/proxy example.com

$ xxd /tmp/request
00000000: 0094 0000 0f00 5345 5256 4552 5f50 524f  ......SERVER_PRO
00000010: 544f 434f 4c08 0048 5454 502f 312e 3109  TOCOL..HTTP/1.1.
00000020: 0050 4154 485f 494e 464f 0100 2f0e 0052  .PATH_INFO../..R
00000030: 4551 5545 5354 5f4d 4554 484f 4403 0047  EQUEST_METHOD..G
00000040: 4554 0b00 5245 5155 4553 545f 5552 4901  ET..REQUEST_URI.
00000050: 002f 0c00 5155 4552 595f 5354 5249 4e47  ./..QUERY_STRING
00000060: 0000 0900 4854 5450 5f48 4f53 540d 0065  ....HTTP_HOST..e
00000070: 7878 7861 6d70 6c65 2e63 6f6d 0b00 5345  xxxample.com..SE
00000080: 5256 4552 5f4e 414d 450d 0065 7878 7861  RVER_NAME..exxxa
00000090: 6d70 6c65 2e63 6f6d 0a                   mple.com.

$ xxd /tmp/response
00000000: 4854 5450 2f31 2e31 2032 3030 204f 4b0d  HTTP/1.1 200 OK.
00000010: 0a43 6f6e 7465 6e74 2d54 7970 653a 2074  .Content-Type: t
...

According to the protocol specification, first comes the header:

struct uwsgi_packet_header {
    uint8_t modifier1;
    uint16_t datasize;
    uint8_t modifier2;
};

In our case modifier1 == 0, datasize == 0x94 (little endian), modifier2 == 0. That makes it:

Standard WSGI request followed by the HTTP request body

Then the vars:

struct uwsgi_var {
    uint16_t key_size;
    uint8_t key[key_size];
    uint16_t val_size;
    uint8_t val[val_size];
}

A key:

00000000:           0f00 5345 5256 4552 5f50 524f      ..SERVER_PRO
00000010: 544f 434f 4c                             TOCOL

A value:

00000010:             08 0048 5454 502f 312e 31         ..HTTP/1.1

A key:

00000010:                                      09                 .
00000020: 0050 4154 485f 494e 464f                 .PATH_INFO

A value:

00000020:                          0100 2f                   ../

And so on until the final variable:

00000070:                               0b00 5345              ..SE
00000080: 5256 4552 5f4e 414d 45                   RVER_NAME

00000080:                       0d 0065 7878 7861           ..exxxa
00000090: 6d70 6c65 2e63 6f6d                      mple.com

The trailing 0a, I guess it was added by socat to separate requests. Then the request length (0x98 bytes) equals the header length (4 bytes) + the length of the vars (0x94 bytes).

Now, how can a tcp socket client [written?] in Python talk with uwsgi? First remove the trailing 0a in /tmp/request, e.g. with vim + xxd. Then:

$ socat TCP4-LISTEN:1234,reuseaddr \
    UNIX-CONNECT:/var/run/uwsgi-socket
$ cat /tmp/request | ncat example.com 1234 | less

(Now socat connects a TCP socket (*:1234) to a Unix domain socket at /var/run/uwsgi-socket. And ncat lets us send the request, that was sent by uwsgi_curl, to port 1234.)

On the other hand, uwsgi can also listen on TCP sockets. So another answer would be, a TCP client can't connect to a Unix domain socket server directly, only via a proxy such as socat.

Answer 2

By talking the uwsgi protocol to a socket. The protocol is described here in the uWSGI documentation.

The Python docs contain a decent tutorial regarding TCP and Unix sockets here.

EDIT: See https://github.com/akx/uwsgi-socket-example for a working example.