CODEWITHSUNDEEP
makefiledependenciesprotocol-buffersautotoolsautomake
bef.tgz
bef.tgz

Reputation: 97

I'm seeing occasional build failure due to auto generated files (automake). How do I create dependencies between autogenerated files?

I have been trying to debug a makefile.am that occasionally causes a build failure in make. In this file, the sources are auto generated .c files and the headers are auto generated .h files. 

..._SOURCES = @buildDirectory@/x.c
              @buildDirectory@/y.c
              @buildDirectory@/z.c

..._HEADERS = @buildDirectory@/x.h
              @buildDirectory@/y.h
              @buildDirectory@/z.h

The failure looks like this

<failedproto>.proto: "symbol1" is not defined.
<failedproto>.proto: "symbol2" is not defined.
<failedproto>.proto: "symbol3" is not defined.
...
<failedproto>.proto: warning: Import <failedproto>.proto but not used.
make: *** [<failedproto>.c] Error 1
make: *** Waiting for unfinished jobs....

All of these symbols appear in the a corresponding .h. This leads me to think that the .c is being generated before the .h, and its just a straight race. I have added both ..._SOURCES and _HEADERS to BUILT_SOURCES, but I still see the failure. So my next instinct is to create a dependency for the .c on the .h. How do I do this, since they are both auto generated? Also, any alternative solutions would be welcome too.

Hopefully my formatting is not confusing.

Edit with some more detail:

These files are being auto generated by the protoc-c compiler: https://github.com/protobuf-c/protobuf-c The protoc-c takes these .proto files and generates .pb-c.c and .pb-c.h files, making me think that these two are not dependent after all. Some in house code is also run, which generates other .proto files, I will call them nameX.proto and nameY.proto, which in turn generate nameX.pb-c.c/nameX.pb-c.h and nameY.pb-c.c/nameY.pb-c.h. A more accurate example of the Makefile.am is like this:

..._SOURCES = @buildDirectory@/name.pb-c.c
              @buildDirectory@/nameX.pb-c.c
              @buildDirectory@/nameY.pb-c.c

..._HEADERS = @buildDirectory@/name.pb-c.h
              @buildDirectory@/nameX.pb-c.h
              @buildDirectory@/nameY.pb-c.h

I have been trying to track these dependencies, and I will try and describe what conclusions I have come to. nameX.pb-c.c includes its corresponding header nameX.pb-c.h. That header includes nameY.pb-c.h, making me think that nameX.proto is being compiled into nameX.pb-c.c/nameX.pb-c.h before nameY.proto can be compiled. Since there is an include relationship between nameX.pb-c.h and nameY.pb-c.h, the build fails because nameX.pb-c.h needs nameY.pb-c.h. This leads me to two rules I've been suspicious about from the start. These rules are generalized like this:

$(OUT_DIRECTORY)/%nameX.proto:$(SRC_DIRECTORY)/name.proto $(SRC_DIRECTORY)/nameY.proto
        command $(OUT_DIRECTORY) $(FLAGS) $<

$(OUT_DIRECTORY)/%nameX.proto:$(SRC_DIRECTORY)/name.proto 
        command $(OUT_DIRECTORY) $(FLAGS) $<

Could this be an issue? What is stopping the second rule from being run if it truly needs the first rule?

To make matters worse, many of the .proto files are intermediate files (they are generated then discarded throughout the build) so I cannot look at them to see what they look like.

Upvotes: 0

Views: 175

Answers (1)

MadScientist
MadScientist

Reputation: 101041

It's very unusual to use @...@ replacements throughout your makefile like this. Normally you would assign the replacement once, to a make variable, then use the variable instead (in addition to being "nicer to read", this allows someone to override this value on the make command line if they want to):

BUILDDIR = @buildDirectory@

..._SOURCES = $(BUILDDIR)/x.c
          $(BUILDDIR)/y.c
          $(BUILDDIR)/z.c

..._HEADERS = $(BUILDDIR)/x.h
          $(BUILDDIR)/y.h
          $(BUILDDIR)/z.h

Also, it seems likely to me that there are standard automake variables that might already cover this value; if so it's better to use the standard ones than invent new ones... but obviously there's no way to know that without knowing more about your environment.

Anyway, for your question we need to know more about this autogeneration operation. What do your rules for autogenerating look like now? Is it really the case that the generation of the .c file can't be done until the .h file is generated? That's unusual.

If you list the output file, the input files, and the command needed then it's pretty simple to write a correct rule.

Upvotes: 1

Related Questions