C to perform HTTPS requests with openssl

Question

I am new to C language and I am trying to perfom a simple SSL GET request with OpenSSL in Linux Ubuntu 18.04. I have this I found at Simple C example of doing an HTTP POST and consuming the response

#include <stdio.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <openssl/bio.h>

#define HOST "www.google.com"
#define PORT "443"

int main() {

    //
    //  Initialize the variables
    //
    BIO* bio;
    SSL* ssl;
    SSL_CTX* ctx;

    //
    //   Registers the SSL/TLS ciphers and digests.
    //
    //   Basically start the security layer.
    //
    SSL_library_init();

    //
    //  Creates a new SSL_CTX object as a framework to establish TLS/SSL
    //  or DTLS enabled connections
    //
    ctx = SSL_CTX_new(SSLv23_client_method());

    //
    //  -> Error check
    //
    if (ctx == NULL)
    {
        printf("Ctx is null\n");
    }

    //
    //   Creates a new BIO chain consisting of an SSL BIO
    //
    bio = BIO_new_ssl_connect(ctx);

    //
    //  Use the variable from the beginning of the file to create a 
    //  string that contains the URL to the site that you want to connect
    //  to while also specifying the port.
    //
    BIO_set_conn_hostname(bio, HOST ":" PORT);

    //
    //   Attempts to connect the supplied BIO
    //
    if(BIO_do_connect(bio) <= 0)
    {
        printf("Failed connection\n");
        return 1;
    }
    else
    {
        printf("Connected\n");
    }

    //
    //  The bare minimum to make a HTTP request.
    //
    char* write_buf = "GET / HTTP/1.1\r\n"
                      "Host: " HOST "\r\n"
                      "Connection: close\r\n"
                      "\r\n";

    //
    //   Attempts to write len bytes from buf to BIO
    //
    if(BIO_write(bio, write_buf, strlen(write_buf)) <= 0)
    {
        //
        //  Handle failed writes here
        //
        if(!BIO_should_retry(bio))
        {
            // Not worth implementing, but worth knowing.
        }

        //
        //  -> Let us know about the failed writes
        //
        printf("Failed write\n");
    }

    //
    //  Variables used to read the response from the server
    //
    int size;
    char buf[1024];

    //
    //  Read the response message
    //
    for(;;)
    {
        //
        //  Get chunks of the response 1023 at the time.
        //
        size = BIO_read(bio, buf, 1023);

        //
        //  If no more data, then exit the loop
        //
        if(size <= 0)
        {
            break;
        }

        //
        //  Terminate the string with a 0, to let know C when the string 
        //  ends.
        //
        buf[size] = 0;

        //
        //  ->  Print out the response
        //
        printf("%s", buf);
    }

    //
    //  Clean after ourselves
    //
    BIO_free_all(bio);
    SSL_CTX_free(ctx);

    return 0;
}

This code works well, but with some sites e.g. ghostbin.co, the connection fails... what is causing this? I compile the code using gcc test.c -o test -lssl -lcrypto. I'm using OpenSSL 1.1.1 11 Sep 2018

Answer 1

Almost always when an OpenSSL routine returns an error indication you can and should get additional information from the error stack; see https://www.openssl.org/docs/faq.html#PROG8 and the man pages e.g.
https://www.openssl.org/docs/manmaster/man3/ERR_print_errors.html
https://www.openssl.org/docs/manmaster/man3/ERR_get_error.html
https://www.openssl.org/docs/manmaster/man3/ERR_error_string.html
https://www.openssl.org/docs/manmaster/man3/ERR_load_crypto_strings.html

However in this case it will only tell you that the server aborted the handshake by sending alert 40, which is one of the less informative alert codes in SSL/TLS. It takes a little knowledge and looking to find ghostbin.co is on CloudFlare, which requires that connections use Server Name Indication aka SNI -- this is true of many SSL/TLS servers this century (since the toplevel IPv4 exhaust) especially those that are shared like a CDN service or shared hosting. The OpenSSL API does allow a client to send SNI, but only using the per-connection SSL object not the SSL_CTX, and with the (quite old and rather creaky) BIO_ssl the SSL object is hidden, so you'll need to BIO_get_ssl(bio,&sslptr) and then SSL_set_tlsext_host_name(sslptr,name) (although I notice the man page gets the const wrong; this is really a macro that uses SSL_ctrl which takes a definitely non-const pointer).