Python C Extension: Linker says 'multiple definition of symbol'

Question

I am currently working on creating a Python extension using C, which includes the definition of a custom object type. In order to maintain a clear and organized codebase, I have chosen to define and implement this object in a separate file. However, as per the Python C API, functions and objects must be defined as static.

Despite this, I have used the __attribute__((visibility("hidden"))) attribute for this object. Unfortunately, I am encountering a recurring error from the linker, specifically, "multiple definition of symbol."

The code provided below is a minimal and easily replicable demonstration of the issue encountered in my larger project. Due to the large number of type objects present in the main project, I am seeking to organize them into separate files for improved maintainability.

My project structure:

.
├── example.c
├── include
│   ├── lcg.h
│   └── macros.h
├── lcg.c
└── setup.py

example.c

#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include "lcg.h"

static PyMethodDef example_methods[] = {
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef example_module = {
    PyModuleDef_HEAD_INIT,
    "example",
    "An example module",
    -1,
    example_methods
};

PyMODINIT_FUNC PyInit_example(void) {
    PyObject *module;

    if (PyType_Ready(&LCG) < 0)
        return NULL;

    module = PyModule_Create(&example_module);

    if (module == NULL)
        return NULL;

    Py_INCREF(&LCG);
    PyModule_AddObject(module, "LCGType", (PyObject*)&LCG);

    return module;
}

lcg.c

SYM_PRIVATE is defined in macros.h and expands to __attribute__((visibility("hidden"))).

#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include "structmember.h"
#include "macros.h"  /* for SYM_PRIVATE */
#include "lcg.h"

static PyObject *LCG__dummy(LCG_Type *self)
{
    int value = 10;
    return PyFloat_FromDouble(value);
}

static PyMemberDef LCG_members[] = {
    {NULL} /* sentinel */
};

static PyMethodDef LCG_methods[] = {
    {"dummy", (PyCFunction)LCG__dummy, METH_NOARGS, "A simple dummy function on LCG module."},
    {NULL} /* sentinel */
};

SYM_PRIVATE PyTypeObject LCG = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "example.LCG",
    .tp_doc = PyDoc_STR("Linear Congruential Generator"),
    .tp_basicsize = sizeof(LCG_Type),
    .tp_itemsize = 0,
    .tp_flags = Py_TPFLAGS_DEFAULT,
    .tp_members = LCG_members,
    .tp_methods = LCG_methods,
};

lcg.h

#ifndef LCG_H
#define LCG_H

#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include "structmember.h"
#include "macros.h"  /* for SYM_PRIVATE */

typedef struct {
    PyObject_HEAD
    int foo;
} LCG_Type;

SYM_PRIVATE PyTypeObject LCG;

#endif /* LCG_H */

setup.py

from setuptools import Extension
from setuptools import setup

example_module = Extension('example', sources=['example.c', 'lcg.c'], include_dirs=['include'])

setup(name = 'example',
      version = '1.0',
      description = 'An example module',
      ext_modules = [example_module]
)

When executing python3.10 setup.py build I got:

running build
running build_ext
building 'example' extension
creating build
creating build/temp.linux-x86_64-cpython-310
x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Iinclude -I/usr/include/python3.10 -c example.c -o build/temp.linux-x86_64-cpython-310/example.o
x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Iinclude -I/usr/include/python3.10 -c lcg.c -o build/temp.linux-x86_64-cpython-310/lcg.o
creating build/lib.linux-x86_64-cpython-310
x86_64-linux-gnu-gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 build/temp.linux-x86_64-cpython-310/example.o build/temp.linux-x86_64-cpython-310/lcg.o -L/usr/lib/x86_64-linux-gnu -o build/lib.linux-x86_64-cpython-310/example.cpython-310-x86_64-linux-gnu.so
/usr/bin/ld: build/temp.linux-x86_64-cpython-310/lcg.o:/home/user/workspaces/python-extension-example/src/include/lcg.h:14: multiple definition of `LCG'; build/temp.linux-x86_64-cpython-310/example.o:/home/user/workspaces/python-extension-example/src/include/lcg.h:14: first defined here
collect2: error: ld returned 1 exit status
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

I am running gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38 and Python 3.10.6.

Answer 1

In lcg.h, comment out:

// SYM_PRIVATE PyTypeObject LCG;

In example.c, add:

extern LCG;

Then you will build the Python module successfully.