Make a static build with a program using sqlite

Question

Context: I'm doing this as an experiment to understand how static vs dynamic linking works, please avoid saying it is stupid or whatever. It has interests and I can't find much about by googling.

TL;DR: go to "The problem" point.

First step: the program. I tried this little program using sqlite, it is stupid, but enough to illustrate.

#include <stdio.h>
#include <sqlite3.h>

int main() {
   printf("Hello, World!\n");
   printf("%s\n", sqlite3_libversion()); 

   return 0;
}

Building: I have the following Makefile

build:
    gcc -o test main.c -lsqlite3

Static build: this is the whole point here. I tried this Makefile.

build.o:
    gcc -c main.c

build.static: build.o
    gcc main.o /usr/lib/x86_64-linux-gnu/libsqlite3.a /usr/lib/x86_64-linux-gnu/libpthread.a -o static-test

Why?

1. I use the binary libsqlite3.a to replace libsqlite (obviously)
2. After trying to compile, I had some issues with pthread, so I added pthread.a binary

The problem: I still get issues... But I now have issues asking for dlopen (and so on). Isn't this supposed to be included by default? Also, it is used to... Include new library dynamically, how can I end this dependency hell?

$ make build.static                   
gcc -c main.c
gcc main.o /usr/lib/x86_64-linux-gnu/libsqlite3.a /usr/lib/x86_64-linux-gnu/libpthread.a -o static-test
/usr/bin/ld : /usr/lib/x86_64-linux-gnu/libsqlite3.a(sqlite3.o) : dans la fonction « fts5Bm25Function » :
(.text+0x2c914) : référence indéfinie vers « log »
/usr/bin/ld : /usr/lib/x86_64-linux-gnu/libsqlite3.a(sqlite3.o) : dans la fonction « unixDlError » :
(.text+0x345e3) : référence indéfinie vers « dlerror »
/usr/bin/ld : /usr/lib/x86_64-linux-gnu/libsqlite3.a(sqlite3.o) : dans la fonction « unixDlClose » :
(.text+0x8358) : référence indéfinie vers « dlclose »
/usr/bin/ld : /usr/lib/x86_64-linux-gnu/libsqlite3.a(sqlite3.o) : dans la fonction « unixDlSym » :
(.text+0x836b) : référence indéfinie vers « dlsym »
/usr/bin/ld : /usr/lib/x86_64-linux-gnu/libsqlite3.a(sqlite3.o) : dans la fonction « unixDlOpen » :
(.text+0x837d) : référence indéfinie vers « dlopen »
/usr/bin/ld : /usr/lib/x86_64-linux-gnu/libpthread.a(pthread_create.o) : dans la fonction « allocate_stack » :
/build/glibc-ZN95T4/glibc-2.31/nptl/allocatestack.c:525 : référence indéfinie vers « _dl_stack_flags »
/usr/bin/ld : /build/glibc-ZN95T4/glibc-2.31/nptl/allocatestack.c:647 : référence indéfinie vers « _dl_stack_flags »
/usr/bin/ld : /usr/lib/x86_64-linux-gnu/libpthread.a(elision-lock.o) : dans la fonction « do_set_elision_enable » :
/build/glibc-ZN95T4/glibc-2.31/nptl/../sysdeps/unix/sysv/linux/x86/elision-conf.c:66 : référence indéfinie vers « _dl_x86_cpu_features »
/usr/bin/ld : /usr/lib/x86_64-linux-gnu/libpthread.a(nptl-init.o) : dans la fonction « __pthread_initialize_minimal_internal » :
/build/glibc-ZN95T4/glibc-2.31/nptl/nptl-init.c:335 : référence indéfinie vers « _dl_pagesize »
/usr/bin/ld : /build/glibc-ZN95T4/glibc-2.31/nptl/nptl-init.c:344 : référence indéfinie vers « _dl_pagesize »
/usr/bin/ld : /build/glibc-ZN95T4/glibc-2.31/nptl/nptl-init.c:360 : référence indéfinie vers « _dl_init_static_tls »
/usr/bin/ld : /build/glibc-ZN95T4/glibc-2.31/nptl/nptl-init.c:362 : référence indéfinie vers « _dl_wait_lookup_done »
/usr/bin/ld : /usr/lib/x86_64-linux-gnu/libpthread.a(nptl-init.o) : dans la fonction « __pthread_get_minstack » :
/build/glibc-ZN95T4/glibc-2.31/nptl/nptl-init.c:393 : référence indéfinie vers « _dl_pagesize »
collect2: error: ld returned 1 exit status
make: *** [Makefile:8 : build.static] Erreur 1

Answer 1

ℹ️ You should take a look at https://stackoverflow.com/a/66094108/2263684 which is really instructive on the topic.

The first solution is to use ldd to see what are the dependencies. Then you need to add them to the compile command like it they are going to be link (they are not going to). This may have issues because the order matters. Here is what was the solution for my case:

gcc -o test main.c libsqlite3.a -lm -ldl -lpthread -static

Another solution probably more reliable is to use pkg-config, it gives the libraries to link and their order to include (in my case) sqlite:

# Gives the name of the package
pkg-config --list-all | grep sqlite
# Give dependencies... In the right order!
pkg-config --static --libs sqlite3

Once you have dependencies, just re-compose the build method with them.

Answer 2

Dynamic libraries contain a list of dependencies to other dynamic libraries. If the dynamic library is loaded, the system will automatically try to locate and also load the dependencies of it.

E.g. if you link against libA and libA depends on libB, your main executable doesn't have to link against libB because libB will automatically be loaded when libA is loaded.

No such thing exists for static libraries. Static libraries are just archives of object files (think of multiple .o files wrappend into a single .tar file). Object files have no list of dependencies. When you link object files to a dynamic library or executable binary, you tell the linker which other libraries are required and from that arguments, the list of dependencies is created.

E.g.

ld -shared -o xyz -l abc -l efg *.o

This call instructs ld to create a dynamic library named xyz and link it against the libraries abc and efg. And in case those two are dynamic libraries, they are added to the dependency list of xyz. Thus whoever loads xyz must then also load abc and efg.

However, object files have not been linked yet. They are the results of compiled source code files and to compile source code files, you don't have to even specify any libraries. Thus object files have no dependencies and archives of object files have neither. You need to ensure that you satisfy all decencies when linking against them.

As static libraries have no dependency list, there is no simple way to find out on which other libraries they depend. If that isn't documented anywhere, you can only look which symbols are missing when you try to link against them and then locate the libraries offering those symbols and link against those as well.

In case you also have a dynamic library version of your static one, you are lucky as then you can just check which dependencies the dynamic library has.

ldd someLibrary

objdump -p someLibrary | grep NEEDED

readelf -d someLibrary | grep NEEDED

All three commands list the direct dependencies, ldd also lists the indirect one (dependencies of the dependencies) when called with --verbose, in case of the others you have ti traverse the entire dependency chain yourself. Pick whichever works on your system or can be installed in case it is missing.

For a real static binary, you need to have static versions of all direct and indirect dependencies and link against all of them.

Fun fact:
Now that you know that static libraries are just archives of object files, it's quite obvious where the file extension .a comes from or why the tool to create them is named ar.