CODEWITHSUNDEEP
csslopenssl
S. A.
S. A.

Reputation: 3754

Directly Read/Write Handshake data with Memory BIO

I need to create an OpenSSL connection where I can directly read/write handshake data. The reason is the handshake data will be transported in a UDP connection (DTLS is not an option, because the data is not directly in the datagram, but inside another protocol packets, EAP if you're curious). So far, I've created an OpenSSL connection but I've not even been able to read the client's handshake to send to the server.

In my research I've found I need a Memory BIO to read/write to the connection, but cannot figure out how to extract the handshake data. Here's how I initialize the client connection:

SSL_library_init();
SSL_load_error_strings();
OpenSSL_add_all_algorithms();

ctx = SSL_CTX_new(SSLv3_client_method());
SSL_CTX_set_verify(ctx, SSL_VERIFY_NONE, NULL);

ssl = SSL_new(ctx);
rbio = BIO_new(BIO_s_mem());
wbio = BIO_new(BIO_s_mem());

SSL_set_bio(ssl, rbio, wbio);
SSL_set_connect_state(ssl);

I've tried doint SSL_connect, to initiate the handshake:

int ret = SSL_connect(ssl);

But returns -1, and doing SSL_get_error(ssl, res) I get an error code 2, then I execute ERR_error_string with that code and get:

error:00000002:lib(0):func(0):system lib

Also, if I use SSL_do_handshake instead of SSL_connect I get exactly the same error.

I've been able to set a OpenSSL connection over TCP, but have never done this with Memory BIOs, so any help with this would be very appreciated. Thanks!

Upvotes: 17

Views: 14220

Answers (2)

S. A.
S. A.

Reputation: 3754

Finally I get it to work, I was in the right way:

The function SSL_set_connect_state(ssl) is needed to tell the connection to be prepared for the handshake init. Then, we call SSL_do_handshake(ssl) to start the handshake. This function will return -1 because the handshake was not finished, but we can actually read from the client ssl connection BIO writer and send the data using the protocol we want (in my case, EAP RADIUS packets over UDP).

Client

ctx = SSL_CTX_new(SSLv3_client_method());
SSL_CTX_set_verify(ctx, SSL_VERIFY_NONE, NULL);

ssl = SSL_new(ctx);
rbio = BIO_new(BIO_s_mem());
wbio = BIO_new(BIO_s_mem());

SSL_set_bio(ssl, rbio, wbio);
SSL_set_connect_state(ssl); 

SSL_do_handshake(ssl); // This will return -1 (error) as the handshake is not finished, we can ignore it.

char buf[4096];
BIO_read(wbio, buf, 4096); // Read from BIO, put data in buffer

// Then use data in buffer to send to the server

The server, in the other hand, should be configured using the credential and private key. Also, instead of SSL_set_connect_state() we should use SSL_set_accept_state() as the server will wait for the client's handshake hello. Then, we simply write the client handshake hello data to the server BIO reader:

Server

ctx = SSL_CTX_new(SSLv3_server_method()); // This is the server!
SSL_CTX_set_verify(ctx, SSL_VERIFY_NONE, NULL);

ssl = SSL_new(ctx);
rbio = BIO_new(BIO_s_mem());
wbio = BIO_new(BIO_s_mem());

SSL_set_bio(ssl, rbio, wbio);
SSL_set_accept_state(ssl); // The server uses SSL_set_accept_state

// Here we get the data from the client suppose it's in the variable buf
// and write it to the connection reader BIO.
BIO_write(rbio, buf, strlen(buf));

if (!SSL_is_init_finished(ssl)) {
    SSL_do_handshake(ssl);
}

We can use the the SSL_is_init_finished(ssl) function to check if the handshake was done, and while it is not done we call SSL_do_handshake(ssl), and then read again from the BIO_writer to send data to the client.

This process between client and server should be done until the connection is done (i.e. SSL_is_init_finished(ssl) returns true).

Then, after the handshake is done, you can send secure data between client/server, by using the SSL_read and SSL_write functions. Hope this short explanation is useful for someone!

Upvotes: 26

user3155701
user3155701

Reputation: 626

Try doing this

// Connect the TCP socket (this is client side)
int sock = tcp_connect(host, port);

// Create BIO around the connected TCP socket 
BIO *sbio = BIO_new_socket(sock, BIO_NOCLOSE);
SSL_set_bio(ssl,sbio,sbio); 

int ret = SSL_connect(ssl);

Think of BIO as a wrap around the socket. For client side, you have to give it a socket that is connected to the desired host and port using connect(...) call. For server side, just use the socket that is returned from accept(...) call.

Upvotes: 1

Related Questions