Adding an external dependency using bazel genrule fails with error `Genrules without outputs don't make sense`

Question

I am trying to use a genrule to fetch and build an external dependency. The setup is as follows.

[Root]/WORKSPACE

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "gsl",
    url = "https://mirror.freedif.org/GNU/gsl/gsl-latest.tar.gz",
    build_file = '@//thirdparty:gsl.BUILD',
)

[Root]/thirdparty/gsl.BUILD

genrule(
    name = "gsl_genrule",
    srcs = glob([
      "gsl-2.5/**/*.c",
      "gsl-2.5/**/*.h",
    ]),
    outs = glob([
      "gsl-2.5/install/lib/*.so",
      "gsl-2.5/install/include/**/*.h",
    ]),
    cmd = "mkdir install && ./configure --prefix=`pwd`/install && make -j && make install",
)

cc_library(
    name = "gsl",
    srcs = [":gsl_genrule"],
    visibility = ["//visibility:public"],
)

[Root]/src/BUILD

cc_binary(
    name = "hello",
    srcs = [
        "hello.cc",
    ],
    deps = [
        "@gsl//:gsl"
    ],
)

In addition I have an empty BUILD file inside [Root]/thirdparty which prevents an error where bazel cannot find gsl.BUILD. [1]

When I run it the genrule fails as follows.

$  bazel build -c dbg --sandbox_debug //src:*
INFO: Invocation ID: a698121f-fc3f-449a-9e36-eb2a5f59e06b
INFO: SHA256 (https://mirror.freedif.org/GNU/gsl/gsl-latest.tar.gz) = 0460ad7c2542caaddc6729762952d345374784100223995eb14d614861f2258d
ERROR: /home/buddhika/.cache/bazel/_bazel_buddhika/3fbc2046978475f6bf9fc76463e16ae5/external/gsl/BUILD.bazel:8:12: in outs attribute of genrule rule @gsl//:gsl_genrule: Genrules without outputs don't make sense
ERROR: Analysis of target '//src:hello.dwp' failed; build aborted: Analysis of target '@gsl//:gsl_genrule' failed; build aborted
INFO: Elapsed time: 11.261s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (3 packages loaded, 1689 targets configured)

I have outputs to be generated by the genrule specified as shown in gsl.BUILD file above. Any idea what I am doing wrong?

[1] https://github.com/bazelbuild/bazel/issues/6873

Answer 1

I'll first focus on your question (but you may want to reconsider both the plan to delegate part of the build to autotools and rely on genrule and generous use of globbing). The error message actually contains your answer:

When bazel analyses the build setup, it will (as instructed) look for all the (existing) "gsl-2.5/install/lib/*.so" and "gsl-2.5/install/include/**/*.h" files and (at the time of analysis) find none. I.e. outs is an empty list and as far as bazel is concerned, your rule indeed produces no output rendering it meaningless genrule.

You could either list actual files you know that would be there. Or you'd have to use a custom rule working with TreeArtifacts (spitting out TreeArtifacts with (all) libraries and headers). You could also pre-build the dependency and depend on the resulting archive/tree. Or "bazelify" its build. (Depending on its nature and how it's used, I'd probably prefer one of the latter two options.)

---

EDIT: a bit more detailed explanation to address the comments.

Bazel first has to analyze and determine what the build tree looks like. When using glob() in outs (which basically doesn't make sense), it will define outs as list of globbed files (at analysis stage) before the rule was executed.

outs = ["concatenated.txt"]

Is fine as it says the rule output is exactly one file concatenated.txt. However:

outs = glob(["*.txt"])

or even:

outs = glob(["concatenated.txt"])

is the same as:

outs = []

because when figuring out what is to be built (and when glob() gets its go) no files match yet (you can think of it as: telling before the build that results of the build will be exactly what find or ls return on output tree, but now before I actually build). The disconnect is, you seem to expect outs to define what to do after the the rule was run (build step took place), but the bazel needs to understand before it executes any rules, what step yields what result and in turn what needs to be done in order to build requested targets.

TreeArtifacts and declare_directory is a bit closer to what you expected above in terms of it tells bazel there will be a directory (with stuff in it) as a result of a rule.

You could also just verbatim list all *.so and *.h files as they would be produced by the build and that would also solve this problem for you.

With pre-built I meant just compile outside of bazel, package the result and refer to an archive with libraries and interface description (headers) from your bazel build. This option would be mostly OK if:

• your dependency does not get updated all the frequently
• you do not have to worry about supporting wide or even unknown range of target system

Otherwise "bazelifying" its build to make it part of your own tree build really could be worth considering.