Python OpenSSL C extension: undefined symbol: AES_set_encrypt_key

Question

I am trying to write to a OpenSSL C extension for Python. The shared library(*.so file) is generated but I am running into undefined symbol errors when importing the module. It throws the following error (undefined symbol: AES_set_encrypt_key):

>>> import openssl_python
Traceback (most recent call last):
  File "", line 1, in 
ImportError: /home/rohith/python_c_extension/aes/openssl_python.cpython-35m-x86_64-linux-gnu.so: undefined symbol: AES_set_encrypt_key

The following is my source code

setup.py

from distutils.core import setup, Extension

openssl_module = Extension('openssl_python',
                       sources = ['openssl_python.c'])

setup(name = 'openssl',
      version = '1.0',
      description = 'Python Package with OpenSSL C Extension',
      ext_modules = [openssl_module])

openssl_python.c

#include 
#include 
#include 
#include 

static PyObject* openssl_module_aes_encrypt(PyObject *self, PyObject *args){
    char* sn;
    if (!PyArg_ParseTuple(args, "s", &sn))
        return NULL;

    AES_KEY       key;
    unsigned char ivec[AES_BLOCK_SIZE];

    unsigned char outBuf[16];

    memcpy(ivec, sqlcFirmwareIvec, sizeof(sqlcFirmwareIvec));

    AES_set_encrypt_key(sqlcFirmwareKey,
                            sizeof(sqlcFirmwareKey) * 8,
                            &key);
    int dataLen = 16;
    int requiredLen = (dataLen / AES_BLOCK_SIZE) * AES_BLOCK_SIZE;

    if (dataLen % AES_BLOCK_SIZE) {
        requiredLen += AES_BLOCK_SIZE;
    }

    AES_cbc_encrypt(sn, outBuf, requiredLen, &key, ivec, AES_ENCRYPT);

    return 1;
}

static PyMethodDef openssl_module_methods[] = { //Can add more functions here
    {   
        "aes_encrypt",
        openssl_module_aes_encrypt,
        METH_VARARGS,
        "Method to encrypt data using Openssl's AES algorithm"
    },  
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef openssl_module_definition = { 
    PyModuleDef_HEAD_INIT,
    "hello_module",
    "A Python module that prints 'hello world' from C code.",
    -1, 
    openssl_module_methods
};

PyMODINIT_FUNC PyInit_openssl_python(void)
{
    Py_Initialize();
    return PyModule_Create(&openssl_module_definition);
}

I compile it using CFLAGS="-lcrypto" python3 ./setup.py build_ext --inplace

Can anyone please help me fix the error?

Thank You. I am not showing the values of key and Ivec on purpose.

EDIT:

Ran the commands: python3 setup.py clean, CFLAGS="-Wl,-z,defs -lcrypto" python3 setup.py build_ext --inplace

This is the output

running build_ext
building 'openssl_python' extension
creating build
creating build/temp.linux-x86_64-3.5
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wl,-z,defs -lcrypto -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.5m -c openssl_python.c -o build/temp.linux-x86_64-3.5/openssl_python.o
openssl_python.c: In function ‘openssl_module_aes_encrypt’:
openssl_python.c:49:21: warning: pointer targets in passing argument 1 of ‘AES_cbc_encrypt’ differ in signedness [-Wpointer-sign]
     AES_cbc_encrypt(sn,
                     ^
In file included from openssl_python.c:4:0:
/usr/include/openssl/aes.h:107:6: note: expected ‘const unsigned char *’ but argument is of type ‘char *’
 void AES_cbc_encrypt(const unsigned char *in, unsigned char *out,
      ^
openssl_python.c:61:12: warning: return makes pointer from integer without a cast [-Wint-conversion]
     return 1;
            ^
openssl_python.c:20:23: warning: unused variable ‘sqlcFirmwarePadding’ [-Wunused-variable]
  static unsigned char sqlcFirmwarePadding[] = {
                       ^
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,defs -lcrypto -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.5/openssl_python.o -o /home/rohith/python_c_extension/aes/openssl_python.cpython-35m-x86_64-linux-gnu.so
build/temp.linux-x86_64-3.5/openssl_python.o: In function `openssl_module_aes_encrypt':
/home/rohith/python_c_extension/aes/openssl_python.c:27: undefined reference to `PyArg_ParseTuple'
/home/rohith/python_c_extension/aes/openssl_python.c:37: undefined reference to `AES_set_encrypt_key'
/home/rohith/python_c_extension/aes/openssl_python.c:49: undefined reference to `AES_cbc_encrypt'
build/temp.linux-x86_64-3.5/openssl_python.o: In function `PyInit_openssl_python':
/home/rohith/python_c_extension/aes/openssl_python.c:84: undefined reference to `Py_Initialize'
/home/rohith/python_c_extension/aes/openssl_python.c:86: undefined reference to `PyModule_Create2'
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

zwol · Accepted Answer

The essential problem here is that when setup.py links your extension it's putting -lcrypto on the command line before the object file with your code in it. The Unix linker processes objects and libraries strictly left to right on the command line: -lcrypto foo.o will not use libcrypto to resolve symbols in foo.o. This is for historical reasons and no longer makes a whole lot of sense but we're stuck with it because it would break too many Makefiles to change it. Also for historical reasons which no longer make a whole lot of sense, if you don't put -Wl,-z,defs on the command line, a shared library (compiled-code Python extensions are technically shared libraries) with undefined symbols in it isn't a link-time error, which is why the build appeared to work.

Your extension intrinsically requires libcrypto. If I am reading the Distutils documentation correctly, that means you should specify it in the libraries= keyword argument to Extension(...) rather than putting it in CFLAGS. Like this:

openssl_module = Extension('openssl_python',
                       sources = ['openssl_python.c'],
                       libraries = ['crypto'])

Python OpenSSL C extension: undefined symbol: AES_set_encrypt_key

Answers (2)

Related Questions