CODEWITHSUNDEEP
bazel
Curious
Curious

Reputation: 2971

How to query list of data files used by a bazel test

I have a cc_test defined as follows:

filegroup(
    name = "test_data",
    srcs = [
        "abc/abc.txt",
        "def.txt",
    ],
)

cc_test(
    name = "my_test",
    size = "small",
    srcs = [
        "test_a.cpp",
    ],
    data = [
        ":test_data",
    ],
)

I would like to query the data files (or runfiles) used for this test (in this case abc/abc.txt and def.txt) using bazel query.

I need a list of data files used by some of my tests to be used in a script.

This is how far I have gotten:

$bazel query 'kind("source file", deps(//xxx/...))'
@bazel_tools//tools/test:test-setup.sh
@bazel_tools//tools/test:collect_coverage.sh
@bazel_tools//tools/def_parser:no_op.bat
@bazel_tools//tools/def_parser:def_parser.exe
@bazel_tools//tools/cpp:link_dynamic_library.sh
@bazel_tools//tools/cpp:grep-includes.sh
@bazel_tools//tools/cpp:build_interface_so
@bazel_tools//tools/coverage:dummy_coverage_report_generator
@bazel_tools//third_party/def_parser:def_parser_main.cc
@bazel_tools//third_party/def_parser:def_parser.h
@bazel_tools//third_party/def_parser:def_parser.cc
//xxx:test_a.cpp
//xxx:def.txt
//xxx:abc/abc.txt

I only want a subset of this list, that is only the test files (//xxx:def.txt and //xxx:abc/abc.txt)

Is is possible to do so?

Upvotes: 2

Views: 7787

Answers (1)

Jin
Jin

Reputation: 13473

I don't think there's a way to do this using bazel query, but we can do this with Aspects.

Aspects in Bazel allow you to traverse the dependency graph through attribute edges (e.g. deps and srcs) and create custom actions.

Example:

Given this BUILD file, we can create a custom rule and aspect to traverse the dependency graph through deps, and collect the data files into a file.

load("//:collect_data_files.bzl", "collect_data_files")

filegroup(
    name = "test_foo_data",
    srcs = [
        "foo.txt",
        "foo/foo.txt",
    ],
)

cc_test(
    name = "test_foo",
    size = "small",
    srcs = [
        "test_foo.cpp",
    ],
    data = [
        ":test_foo_data",
    ],
)

filegroup(
    name = "test_bar_data",
    srcs = [
        "bar.txt",
        "bar/bar.txt",
    ],
)

filegroup(
    name = "test_bar_lib_data",
    srcs = [
        "bar_lib.txt",
        "bar_lib/bar_lib.txt",
    ],
)

cc_library(
    name = "test_bar_lib",
    srcs = ["test_bar_lib.cpp"],
    data = [":test_bar_lib_data"],
)

cc_test(
    name = "test_bar",
    size = "small",
    srcs = [
        "test_bar.cpp",
    ],
    data = [
        ":test_bar_data",
    ],
    deps = [":test_bar_lib"],
)

collect_data_files(
    name = "collect_data",
    testonly = 1,
    deps = [
        "test_bar",
        "test_foo",
    ],
)

This is the dependency graph:

enter image description here

We have two tests, test_foo and test_bar, which depend on data files. test_bar also depends on a cc_library that has its own data files.

In collect_data_files.bzl, we create an aspect to collect the data files on targets through the deps attribute transitively.

DataFilesInfo = provider(
    fields = {
        "data_files": "Data files for this target",
    },
)

def _collect_data_aspect_impl(target, ctx):
    data_files = []
    if hasattr(ctx.rule.attr, "data"):
        for src in ctx.rule.attr.data:
            for f in src.files:
                data_files += [f]
    for dep in ctx.rule.attr.deps:
        data_files += dep[DataFilesInfo].data_files
    return [DataFilesInfo(data_files = data_files)]

collect_data_aspect = aspect(
    attr_aspects = [
        "deps",
    ],
    implementation = _collect_data_aspect_impl,
)

Then in the same file, we can define the collect_data_files rule to write the list of target labels and their data files into a file.

def _collect_data_rule_impl(ctx):
    data_files_string = ""
    for dep in ctx.attr.deps:
        data_files = [f.path for f in dep[DataFilesInfo].data_files]
        data_files_string += str(dep.label) + ","
        data_files_string += ",".join(data_files) + "\n"
    ctx.actions.write(ctx.outputs.data_files, data_files_string)

collect_data_files = rule(
    attrs = {
        "deps": attr.label_list(aspects = [collect_data_aspect]),
    },
    outputs = {
        "data_files": "%{name}_data_files.txt",
    },
    implementation = _collect_data_rule_impl,
)

Now, we can build the collect_data_files target:

$ bazel build :collect_data
INFO: Analysed target //:collect_data (10 packages loaded).
INFO: Found 1 target...
Target //:collect_data up-to-date:
  bazel-bin/collect_data_data_files.txt
INFO: Elapsed time: 1.966s, Critical Path: 0.01s
INFO: 0 processes.
INFO: Build completed successfully, 2 total actions

The results are written into the output file:

$ cat bazel-bin/collect_data_data_files.txt
//:test_bar,bar.txt,bar/bar.txt,bar_lib.txt,bar_lib/bar_lib.txt
//:test_foo,foo.txt,foo/foo.txt

Upvotes: 4

Related Questions