placing object files in different directories in nonrecursive build

Question

Background

We build .mex files for both Matlab and Octave. The source code is common to both Matlab and Octave but with slight modifications here and there (taken care of via macro conditionals in the code). For reference, the source structure is:

                      mex/
                       |
              ------------------------------
              |                            |
           sources/                      build/
              |                            |
        -----------            -------------------------------
        |         |            |          |        |         |
       mex1/ ... mexN/       octave/  mex1.am ... mexN.am  matlab/
        |                      |                             |
    -----------           -------------                -------------
    |         |           |           |                |           |
file1.cc ... fileM.cc  configure.ac  Makefile.am   configure.ac  Makefile.am

The .mex source files are located in mex/sources/mex*/ and we compile multiple .mex files.

The Matlab build system is located in mex/build/matlab/, where we run './configure' to locate the headers/libraries needed to link against for the compilation of Matlab .mex files. Analogous configure.ac and Makefile.am files exist in /mex/build/octave/, modified to find the Octave headers and libraries.

The automake build instructions (the paths to the source files, the .mex files to be built) common to both Matlab and Octave exist in the files named /mex/build/mex*.am and typically look like:

mexI_PROGRAMS += mexI

nodist_mexI_SOURCES = \
    ../../sources/mexI/mexI.c \
    ../../sources/mexI/mexI_file1.c \
    ../../sources/mexI/mexI_fileN.c

Recursive make worked perfectly for our setup (we used to have directories named /mex/build/matlab/mex*/ and /mex/build/octave/mex*/ that contained Makefile.am files; object files would be compiled there and the output files would be located there). Now we are moving to a non-recursive build given the changes that GNU subdir-objects change. Starting with the Matlab build, I was able to modify it to work non-recursively without any problems.

The Problem

In moving to the non-recursive build, object files are now produced in the same directories as their source files, i.e., /mex/sources/mex*/. As I have already compiled the Matlab .mex files, the *.o files exist in these source directories. So, when I go compile the Octave files, make skips directly to the linking step as it already sees the recent *.o files in the directory. The problem is, of course, that these *.o files were compiled with Matlab headers and hence will not work in an Octave build.

The easiest solution would be to simply define a directory in which the object file would be placed. Currently, a typical build compilation command looks like (removing flags):

gcc ... -o ../../sources/mex1/mex1.o ../../sources/mex1/mex1.c

I would just like to change the location of the file passed to the -o flag, but don't see how to do this in automake. Unless, of course, there's a more elegant solution.

Answer 1

I would just like to change the location of the file passed to the -o flag, but don't see how to do this in automake. Unless, of course, there's a more elegant solution.

I observe in the first place that the compiled object files are being placed next to the source files because that's what you said to do by giving Automake the subdir-objects option and performing an in-source build (or in fact two). There are several options for having them placed somewhere else, but two of them stand out, as they are tied directly to the reasons for the problem:

1. Remove the subdir-objects Automake option. Your intermediate .o files should then end up in the build directory from which you run make. This is a little messy, but that should not be a concern, as the generated Makefile knows which files they are, and can therefore distinguish them from the installable products where necessary. The main issue here is that you could run into trouble with collisions among the object files associated with different .mex files built for the same target.

2. Perform out-of-source instead of in-source builds, one for each main target (octave / matlab), each in its own build tree. Contrary to what you may think, that's not what you're doing now, at least not from the Autotools' perspective. When you perform an out-of-source build, all generated files are written to the build tree, so having separate build trees for octave and matlab should resolve your issue cleanly. This is the option I recommend.

The Autotools provide support for out-of-source building automatically. All you need to do is create the desired build directory, make that your working directory, and then run the project's configure script from there. The build directory can be a subdirectory of the source root if you wish, though it doesn't have to be. The key thing to understand is the the "top source directory" is the one containing the configure script, and the "top build directory" is the one that was the working directory from which you ran that script. Thus, if you were in the project root, you might do this:

$ cd build
$ mkdir build-octave build-matlab

$ cd build-octave
$ ../octave/configure
$ make
$ make install

$ cd ../build-matlab
$ ../matlab/configure
$ make
$ make install

But it is possible to write your Makefile.am files in ways that break that support by assuming in-source building. The Makefile.am excerpt you present does precisely that by depending on a specific relative path from the build directory to the sources. If your Makefile.am wants to refer to a source file (as opposed to a built file) relative to the top source directory, then it can use the automatic $(top_srcdir) variable provided by Automake. (And that will not interfere with in-source building, either.) For example, you ought to be able to do this:

nodist_mexI_SOURCES = \
    $(top_srcdir)/../../sources/mexI/mexI.c \
    $(top_srcdir)/../../sources/mexI/mexI_file1.c \
    $(top_srcdir)/../../sources/mexI/mexI_fileN.c

I don't so much like referring to resources above the top source directory, but it ought to work.

---

Update:

Consider also modifying the Autotooling to have only one build system for both target types. Autoconf supports defining your own --enable and --with options, which you can use to select the target type at the time you run configure. Autoconf and Automake support conditionals of various kinds that can use these and other criteria to adjust the details of the build. This could be a big win over maintaining parallel build systems separately. This could all work together very nicely with out-of-source building.