Send http URL through C code using visual studios in Windows

Question

I have a video encoder which can start/stop recording by sending a URL.

Start recording: http://192.168.1.5/blah-blah/record/record.cgi

Stop recording: http://192.168.1.5/blah-blah/record/stop.cgi

I don't have to worry about getting a response. I just have to send it.

I was exploring the GET method and also trying to use HappyHTTP library. Is there a simple way to do it without using any library?

I am using visual studios on Windows.

Answer 1

The barebones way of sending a request to server is """pretty simple""".

Firstly, I'll be assuming you know the exact host and port of the url. I assume this since I cannot access the link you've provided. In most cases, you know the host being the domain you type in the url and the port being 80 for http, or 443 for https - I cannot confirm this for the given link however.

In any case, first things first - the headers you'd require on windows for socket programming is-

#include <WinSock2.h>
#include <WS2tcpip.h>

You'll also need to link the winsock2 library, you can do this by putting #pragma comment(lib, "Ws2_32.lib") after your headers. Or you can put Ws2_32.lib in Project Configuration -> Linker -> Input -> Additional Dependencies in Visual Studio.

Now, winsock2 requires you to initiate the library (and also to close it). You can do this in your main function ideally, since the startup is required for any socket connection to be made-

 /* Initialize wsock2 2.2 */
WSADATA wsadat;
if (WSAStartup(MAKEWORD(2, 2), &wsadat) != 0)
{
    printf("WSAStartup failed with error %d\n", WSAGetLastError());
    return 1;
}

And after everything is done-

/* Cleanup wsock2 */
if (WSACleanup() == SOCKET_ERROR)
{
    printf("WSACleanup failed with error %d\n", WSAGetLastError());
    return 1;
}

Now you need to obtain information about the server you want to connect to, we do that using getaddrinfo. (Relevant posix docs)

static struct addrinfo const hints = { .ai_family = AF_UNSPEC, .ai_socktype = SOCK_STREAM };

struct addrinfo* addrs;
int gai_retcode = getaddrinfo("192.168.1.5", "http", &hints, &addrs);
if (gai_retcode != 0)
{
    fprintf(stderr, "Error encountered during getaddrinfo: %s\n", gai_strerrorA(gai_retcode));
    return 1;
}

Let's walk through this a bit, getaddrinfo takes a node parameter - which is either a hostname string or an ip address string.

The second parameter is service - which is a string specifying the port to connect to, or the service being used (http, https, telnet etc - you can find a list of services in the IANA port list or in /etc/services of your linux box).

The third parameter is the address of an addrinfo struct containing "hints" to aid in selecting the right address. For connecting to a server, you usually need to set only the ai_family and ai_socktype members, everything else should be 0 initialized (which is what the compound initializer does automatically). AF_UNSPEC on ai_family means "find me either ipv4 or ipv6 addresses - no particular requirement", SOCK_STREAM on ai_socktype means "find me stream servers addresses" (aka TCP servers, not Datagram servers). HTTP is a stream based protocol.

The final parameter is the address of a pointer to an addrinfo struct. This is where the results of the lookup are stored in.

getaddrinfo returns 0 on success and fills up the pointer to addrs pointer. addrs is now a linked list of addresses returned from the lookup - the first one in the list is directly accessible, the next one is accessible through .ai_next and so on until you reach NULL. The regular shenanigans.

For websites having multiple servers around the world, getaddrinfo will return many addresses. They are arranged in the most relevant way determined by your machine. That is, the first one is the address most likely to work well. But you should ideally try the next ones in the linked list in case the first one fails.

For non zero returns, you can use gai_strerror to print the error message. This is named gai_strerrorA on windows for whatever reason. There's also a gai_strerror on windows but it returns a WCHAR* as opposed to a char*. On linux, you'll be using gai_strerror, not gai_strerrorA. Relevant linux docs on gai_strerror

Now that you have a list of addresses, you can create a socket and connect to one of them-

int sfd = socket(addrs->ai_family, addrs->ai_socktype, addrs->ai_protocol);
if (sfd == INVALID_SOCKET)
{
    fprintf(stderr, "Error encountered in socket call: %d\n", WSAGetLastError());
    return 1;
}
if ((connect(sfd, addrs->ai_addr, addrs->ai_addrlen)) == -1)
{
    fprintf(stderr, "Error encountered in socket call: %d\n", WSAGetLastError());
    closesocket(sfd);
    return 1;
}

NOTE: This is only trying the first address in the linked list, if the first one fails - it just gives up. This isn't ideal for a real world usecase. But for your specific usecase, it probably won't matter.

Pretty self explanatory, socket takes the ai_family, ai_socktype and ai_protocol values from your addrinfo structure and creates a socket. It returns INVALID_SOCKET on failure (this is just -1 on linux). Relevant linux docs on socket.

connect takes the socket descriptor you just got from socket, and the ai_addr and ai_addrlen values from your addrinfo structure to connect to the server. It returns SOCKET_ERROR on failure (again, just -1 on linux). Relevant linux docs on socket.

Now all the low level socket shenanigans is done. Next comes HTTP shenanigans. To make an HTTP request, you need to know the HTTP spec, you can read the entire thing here.

Just kidding, here's a more digestible version - courtesy of MDN.

But basically, the format of a message is sort of like this-

<HTTP-VERB> <REQUEST-PATH> HTTP/<VERSION>\r\n[HEADERS-TERMINATED-WITH-CRLF]\r\n[BODY]

So a GET request to /blah-blah/record/record.cgi using HTTP 1.0 and no headers (no body either, since this is a GET request) looks like-

GET /blah-blah/record/record.cgi HTTP/1.0\r\n\r\n

Before moving on, I need to mention why I'm demonstrating HTTP/1.0 instead of the more sensible for the times - HTTP/1.1. Since not all servers actually support 1.0 fully. The real difference between the 2 is that 1.1 requires a Host: header, and it allows keeping the connection alive - which is more efficient for multiple requests to the same server.

About the Host: header, if you know what hostname the server is expecting in the request header, great! Change it to HTTP/1.1 and put the Host: hostname in. Where hostname is...well, the hostname. For all I know, your server might just be accepting 192.168.1.5 as the hostname, but it also may not.

The second part is the keep-alive connection part. A keep-alive connection gets a bit complicated with blocking sockets. If you decide to use a keep-alive connection and try to recv the response from the server, when there is nothing more to read from the stream, recv will keep blocking until there's something - since the connection is still alive. There are ways to solve this ofcourse, the behaviors of which vary from linux to windows. So I'd recommend just setting the Connection header to close if using HTTP/1.1. I assume this would be "good enough" for your usecase. The HTTP request would then look like-

GET /blah-blah/record/record.cgi HTTP/1.1\r\nHost: hostname\r\nConnection: close\r\n\r\n

Now, to send the request-

char const reqmsg[] = "GET /blah-blah/record/record.cgi HTTP/1.0\r\n\r\n";
if (send(sfd, reqmsg, sizeof reqmsg, 0) == SOCKET_ERROR)
{
    fprintf(stderr, "Error encountered in send call: %d\n", WSAGetLastError());
    closesocket(sfd);
    return 1;
}

send takes the socket descriptor, the message, and the message length and returns the number of bytes sent or SOCKET_ERROR on failure (-1 on linux). Relevant docs for linux on send

There's some peculiarity here that should be noted. send may send less bytes than the length of the full message. This is normal for stream sockets. When this happens, you have to send the rest by calling send again and starting from where send left off (add the return value to the message string pointer to obtain the continuation point). This is omitted for brevity but it is highly unlikely to actually happen for such a short message. If you have any machine from the last decade - send will send that tiny message in full.

Please don't rely on that in an actual project though.

Congrats! you've successfully sent a request to the server. Now you can receive its response. The response format is like this-

HTTP/<VERSION> <STATUS-CODE> <STATUS-MSG>\r\n[HEADERS-TERMINATED-WITH-CRLF]\r\n[BODY]

You don't have to receive if you don't want to, the data will just be sitting there in the stream buffer, which will "vanish" when you close the socket. But here's a small example on parsing out just the status code-

int status;
char stats[13];
if (recv(sfd, stats, 13, 0) != 13)
{
    fprintf(stderr, "Error encountered in recv call: %d\n", WSAGetLastError());
    closesocket(sfd);
    return 1;
}
sscanf(stats, "HTTP/1.0 %d ", &status);

recv, works much like send, just the other way around. It fills the buffer you pass as its second parameter and the maximum number of bytes it'll read can be mentioned in its third parameter. The return value of recv is the number of bytes read. Relevant docs for linux on recv

The peculiarity mentioned before is present here too. recv may read less than the bytes you mention in the third param. But with any half decent internet connection, 13 bytes will definitely be read in one go.

Why 13 bytes? That's the length of the HTTP/1.0 + a space + 3 digit status code + a space. We're only interested in the status code for this. sscanf will then parse out the status code (assuming the server responded correctly) and put it into status.

If you want to read any marginally large size (and/or execute multiple recv calls), be sure to do it in a loop and use the return value of the number of bytes read to appropriately use the buffer.

After all of this, be sure to call freeaddrinfo on addrs and closesocket on sfd.

Further reading: MS docs on Winsock2

On linux, everything is very similar - but with much less headaches :) - I recommend checking out beej's guide to network programming if you're looking for implementing this on linux.

(In fact, it's a great read for windows programmers as well! Up to date with modern functions that support ipv4 and ipv6 seamlessly, no manual and painful struct filling etc)

Everything will be super similar, you just need to change the headers - get rid of the stupid windows specific startups and replace WSAGetLastError inside fprintf to just a regular perror.

Edit: Replacement for compound literal (static struct addrinfo const hints = { .ai_family = AF_UNSPEC, .ai_socktype = SOCK_STREAM };)-

struct addrinfo hints;
memset(&hints, 0, sizeof hints);
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;

Suffice to say, if you have access to a compiler that supports C99 and above - use compound literals instead - it's plain better :)

Answer 2

You can use TCP connection to the server, after connected, you can send the plain HTTP request. The following program shows this, connecting to google with TCP protocol, sending HTTP get request, and getting the first 2K of the response.

Note: See @Chase great comments below, the program is not complete: usage of gethostbyname should be avoided, as this method is deprecated, error handling should be changed to use WSAGetLastError and not errno, the program should call WSACleanup before it ends.

I don't have time now to fix the program, and I leave it as it, as someone may benefit from it as it is now.

#define _CRT_SECURE_NO_WARNINGS
#include <Windows.h>
#include <winsock.h>
#include <stdio.h>

#pragma comment(lib, "Ws2_32.lib")

#define SERVER_PORT ((u_short )80)
#define SERVER_HOST "www.google.com"

int main(int argc, char* argv[])
{
    int ret;
    SOCKET sd;
    struct hostent* he = { 0 };
    struct sockaddr_in addr_info = { 0 };
        
    WSADATA wsaData = { 0 };
    WSAStartup(MAKEWORD(2, 2), &wsaData);

    he = gethostbyname(SERVER_HOST);
    if (he == NULL)
    {
        printf("failed to get host information for '%s': %s\n", SERVER_HOST, strerror(errno));
        exit(-1);
    }

    sd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
    if (sd == INVALID_SOCKET)
    {
        printf("failed to create socket: %s\n", strerror(errno));
        exit(-1);
    }
    
    addr_info.sin_family = AF_INET;      
    addr_info.sin_port = htons(SERVER_PORT);    
    addr_info.sin_addr = *((struct in_addr*)he->h_addr);
    memset(&(addr_info.sin_zero), 0, sizeof(addr_info.sin_zero));     

    printf("connecting to %s ....\n", SERVER_HOST );
    ret = connect(sd, (struct sockaddr*)&addr_info, sizeof(struct sockaddr));
    if ( ret == -1)
    {
        printf("failed to connect: %s\n", strerror(errno));
        exit(-1);
    }

    printf("sending GET /index.html request ....\n");
    char send_buffer[] = "GET /index.html HTTP/1.1\r\nHost : " SERVER_HOST "\r\n\r\n";
    ret = send(sd, send_buffer, sizeof(send_buffer), 0);
    if ( ret == -1)
    {
        printf("failed to send request: %s\n", strerror(errno));
        exit(-1);
    }

    printf("getting response (first 2K)\n");
    char response_buffer[2048] = "";
    //using sizeof(response_buffer)-1 to have the buffer terminate with zero, even if it is fully consumed by the recv call. this is because I want to print it later on.
    ret = recv(sd, response_buffer, sizeof(response_buffer)-1, 0);
    if (ret == -1)
    {
        printf("failed to get response: %s\n", strerror(errno));
        exit(-1);
    }

    printf("received (first 2KB): %s\n", response_buffer);

    closesocket(sd);

    return 0;
}

program output is:

connecting to www.google.com ....
sending GET /index.html request ....
getting response (first 2K)
received: HTTP/1.1 200 OK
Date: Tue, 29 Dec 2020 08:56:34 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2020-12-29-08; expires=Thu, 28-Jan-2021 08:56:34 GMT; path=/; domain=.google.com; Secure
Set-Cookie: NID=205=a7W2c7M39ojimAWRRgn7nwedmdouUrZfo8uBa1wJeEwo8DUk7ibclM-xwp5ozhKO2BYmcRnQ1l4wwjb_DYfOVsDoi-UdtqBmgySL_KlcG6zMjembghO8OL81e2iHee0_cnlDZvSCCGvPnaC0LHNzFtqeWaYSELF7-1t5Khuv4Yc; expires=Wed, 30-Jun-2021 08:56:34 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
Transfer-Encoding: chunked

...