CODEWITHSUNDEEP
csqlite
Thomas Kowalski
Thomas Kowalski

Reputation: 2184

Compiling SQLite without doing the amalgamation

I'm currently trying to add changes in how the SQLite virtual machine executes its code. To do that, I edit the vdbe.c file from the SQLite source.

The issue is, compiling SQLite consists in generating two huge implementation and header files (sqlite3.c and sqlite3.h) by amalgamating several smaller ones, after parsing some of them to generate code and documentation.

Unfortunately, the amalgamation process takes a relatively long time (about 15 seconds). I was wondering if there would be a somewhat easy way to not compile everything every time like it currently is the case, and possibly save a lot of compile time.

The main difficulty stems from the fact that source files are not valid by themselves (they can only compile once they have been amalgamated so that some types have already been defined earlier in the amalgamated file). After several attempts with a simple hand-written Python script (that would simply extract the virtual machine execution code from the amalgamation and keep the rest together), I came to the conclusion that there are two many edge cases to do it this way. I don't really know how to proceed.

Any suggestions are welcome.

Upvotes: 1

Views: 500

Answers (2)

Thomas Kowalski
Thomas Kowalski

Reputation: 2184

So in case anyone needs the answer in the future, here is how I did. You can find the whole discussion on SQLite Forums.

After getting the source:

  • I have a first make pass where I don't change any file (to compile lemon, possibly among others):
make -f Makefile.linux-gcc
  • Then, I edit Makefile.linux-gcc and replace all occurrences of gcc with gcc -x none
  • I change main.mk and change the executable target by adding -lstdc++ at the end:
sqlite3$(EXE):  shell.c libsqlite3.a sqlite3.h
    $(TCCX) $(READLINE_FLAGS) -o sqlite3$(EXE) $(SHELL_OPT) \
        shell.c libsqlite3.a $(LIBREADLINE) $(TLIBS) $(THREADLIB) -lstdc++
  • I run make again using make -f Makefile.linux-gcc and get my sqlite3 executable as expected.

Upvotes: 1

Kuba hasn't forgotten Monica
Kuba hasn't forgotten Monica

Reputation: 98425

I'd say: check in the amalgamation into your source code repository, treat it as your own artifact, and work on it. Whenever you wish to update the amalgamation, use git to help you.

  1. Create two branches sqlite-upstream and sqlite-local.

  2. Check in upstream amalgamation "v1" to sqlite upstream, then merge that to sqlite-local and do whatever local changes you need in that branch.

  3. When upstream releases "v2", commit that to sqlite-upstream.

  4. Merge or rebase - you may have some conflicts to resolve, but those will be much easier to deal with than manual change tracking.

    • Merge sqlite-upstream into sqlite-local, or

    • Duplicate sqlite-local into sqlite-local-v2, then rebase it onto sqlite-upstream, then use sqlite from that branch in dependent code.

Upvotes: 1

Related Questions