CODEWITHSUNDEEP
c++cmake
Steve Lorimer
Steve Lorimer

Reputation: 28679

transitive add_custom_command dependencies not carried over

I have some protobuf files which are in a directory outside of my cmake project (they live in a root directory next to the ${CMAKE_SOURCE_DIR} where our main CMakeLists.txt file is).

root/
+--- src/
|    +--- CMakeLists.txt            # main CMakeLists
|    +--- messages/
|         +--- CMakeLists.txt       # protocol generation CMakeLists
|         +--- base/
|              +--- header.pb.cc
|              +--- header.pb.h
+--- protocols/
     +--- base/
     |    +--- header.proto         # protocol definitions
     .    .

My goal is to create a custom_command which will generate the .pb.cc and .pb.h files and link them into a library for use in my project.

So in the above example, root/src/messages/base/header.pb.cc is created from root/protocols/base/header.proto.

Protobuf generation:

I have created a function (protoc) which loops over a list of protobuf .proto source files, and calls add_custom_command to invoke the protobuf compiler on each file. 

function(protoc SRCS_OUT HDRS_OUT)

    set(options)
    set(values  CPP_OUT CWD)
    set(lists   PROTO)
    cmake_parse_arguments(ARG "${options}" "${values}" "${lists}" "${ARGN}")

    set(GENERATED_SRCS)
    set(GENERATED_HDRS)

    foreach(FILE ${ARG_PROTO})

        # find the absolute path to the .proto file
        get_filename_component(ABS_FILE ${FILE} ABSOLUTE)

        # replace source-dir with dest-dir and generate the path where the 
        #  .pb.cc and .pb.h files will be created
        string(REPLACE ${ARG_CWD} ${ARG_CPP_OUT} PROTO_DEST ${ABS_FILE})

        get_filename_component(FILE_WE  ${PROTO_DEST} NAME_WE)
        get_filename_component(DEST_DIR ${PROTO_DEST} DIRECTORY)

        set(GENERATED_SRC "${DEST_DIR}/${FILE_WE}.pb.cc")
        set(GENERATED_HDR "${DEST_DIR}/${FILE_WE}.pb.h")

        # run the protoc compiler on the .proto file, specifying the generated 
        #  .pb.cc and .pb.h files as OUTPUT, and the .proto file as the
        #  MAIN_DEPENDENCY
        add_custom_command(
            OUTPUT
                ${GENERATED_SRC}
                ${GENERATED_HDR}

            COMMAND
                protoc

            ARGS
                --cpp_out ${ARG_CPP_OUT} ${ABS_FILE}

            WORKING_DIRECTORY
                ${ARG_CWD}

            MAIN_DEPENDENCY
                ${ABS_FILE}

            COMMENT
                "Running C++ protocol buffer compiler on ${FILE}"

            VERBATIM
            )

        set_source_files_properties(${GENERATED_SRC} ${GENERATED_HDR} 
            PROPERTIES GENERATED TRUE)

        list(APPEND GENERATED_SRCS ${GENERATED_SRC})
        list(APPEND GENERATED_HDRS ${GENERATED_HDR})

    endforeach()

    set(${SRCS_OUT} ${GENERATED_SRCS} PARENT_SCOPE)
    set(${HDRS_OUT} ${GENERATED_HDRS} PARENT_SCOPE)

endfunction()

Note that I have specified the dependencies between the .pb.cc and .pb.h files and the .proto files:

I then use the above protoc function to create a library consisting of the generated .pb.cc files:

root/src/messages/CMakeLists.txt:

get_filename_component(PROTOCOLS_DIR "${CMAKE_SOURCE_DIR}/../protocols" ABSOLUTE)

protoc(
    PROTO_SRCS
    PROTO_HDRS

    PROTO
        ${PROTOCOLS_DIR}/base/header.proto

    CWD
        ${PROTOCOLS_DIR}

    CPP_OUT
        ${CMAKE_CURRENT_SOURCE_DIR}
        )

add_library(
    msg_base
    STATIC
    ${PROTO_SRCS} ${PROTO_HDRS}
    )

As such, the command to protoc will have as its PROTO argument

and msg_base will have as its sources

My expectation is that now any target which links msg_base will transitively have a dependency on the .proto files

target -> msg_base -> header.pb.cc -> header.proto

eg:

add_executable(foo ${FOO_SRCS})
target_link_libraries(foo msg_base) # transitive dependency on header.proto

The problem:

On occasion header.proto will be out of date, but foo will link against an old version of msg_base.

msg_base won't be rebuilt with the new header.pb.cc files before foo is linked (with an out of date msg_base).

Questions:

I ask this because the help for add_custom_command does make reference to targets created in the same directory, and I'm wondering if this falls foul of that?

Upvotes: 0

Views: 251

Answers (1)

Steve Lorimer
Steve Lorimer

Reputation: 28679

Research led me to a post on the cmake wiki titled "How can I add a dependency to a source file which is generated in a subdirectory?"

It turns out that cmake currently doesn't know how to link generated files as dependencies to targets in a different directory.

That sounds very much like it is the root cause of this issue.

There is an open bug report / feature request for this, but as of writing it is still open.

The "fix" is to move the creation of each library which includes the generated protobuf files into the subdirectory where those files are generated.

That is, move the creation of msg_base from root/src/messages/CMakeLists.txt to root/src/messages/base/CMakeLists.txt

The resulting tree will look as follows:

root/
+--- src/
|    +--- CMakeLists.txt            # main CMakeLists
|    +--- messages/
|         +--- base/
|              +--- CMakeLists.txt  # protocol generation CMakeLists
|              +--- header.pb.cc
|              +--- header.pb.h
+--- protocols/
     +--- base/
     |    +--- header.proto         # protocol definitions
     .    .

Upvotes: 1

Related Questions