CODEWITHSUNDEEP
sconsbuild-dependenciesincremental-build
Evgen
Evgen

Reputation: 194

How can I get all the source dependencies SCons computed for a given target?

I want to do this programmatically right after a given target is built, during SCons build run, not with --tree or any other command to scons. I have a target node. It might have had some explicit dependencies, used scanners, file extension-based scanners, and whatever else SCons calculated. So like:

all_source_nodes = tgt_node.get_all_sources(...)

I searched the docs and the APIs. Only saw get_stored_implicit on FS nodes. I get None on that and also for .prerequisites and .implicit Node members.

I also found that .sources Node member shows direct sources that were passed into the builder. That's not enough too, of course, because I need essentially all the nodes of the dependency sub-tree, which is a lot more.

Upvotes: 0

Views: 692

Answers (2)

bdbaddog
bdbaddog

Reputation: 3511

You won't get most of that information at the time the SConstruct/SConscript is processed.

The dependency graph is fully populated after that.

Likely you can use sconsign for that. It reads the build database after SCons has completed and you can query for a specific target.

Upvotes: 0

Evgen
Evgen

Reputation: 194

Eureka! :) A combination of build info data and recursive scanner application got me everything scons --tree=prune reports. Full code below, along with some informational messages.

__FULL_DEPS = {}    # all dependencies for the node and its children recursively
__DIRECT_DEPS = {}  # only the dependencies obtained from SCons for the node itself (direct and scanned)

# Find scons deps for this target - these are all in the subtree (same as __FULL_DEPS[tgt])
scons_deps = find_deps(tgt, None, env)
tgt_deps = set(map(str, scons_deps))    # Some nodes refer to the same paths

def find_deps(node, children_func, env):
    """Recursively traverses children dependencies from build info and by applying 
children_func (scanner) to collect the full set of all dependencies 
starting with 'node' as the dependency tree root.
:Parameters:
    - node          the current target we check (root node of the sub-tree)
    - children_func the function called to get the children of a node. May be None.
    - env           the current environment
:Return:
    The set of all source nodes in the dependency sub-tree
    While the node objects are unique, the underlying files/dirs may repeat on rare occasions.
    Collects the global __DIRECT_DEPS and __FULL_DEPS caches
    """
    if node in __FULL_DEPS:
        print("_find_deps '{}' deps - from CACHE: {}."
                            .format(str(node), len(__FULL_DEPS[node])))
        return __FULL_DEPS[node]
    children = set(node.all_children())  # children may repeat
    
    # Apply children func, if available and merge in the results
    scanned_deps = set()
    if children_func:
        scanned_deps = set(children_func(node))
        children |= scanned_deps
    __DIRECT_DEPS[node] = children

    # Iterate all the unvisited children and recursively call find_deps with children_func 
    # specific to each child.
    # FYI: Scanner use code is based on SCons code in Node.render_include_tree()
    subtree_deps = set()
    for child in sorted(children):
        scanner = node.get_source_scanner(child)
        # Note: get_build_scanner_path fails on nodes without executor
        scanner_path = node.get_build_scanner_path(scanner) if (scanner and node.get_executor()) else None
        def children_func(node, env=env, scanner=scanner, path=None):
            return node.get_found_includes(env, scanner, path)
    
        subtree_deps |= find_deps(child, children_func, env)
        __FULL_DEPS[node] = children | subtree_deps

    print("_find_deps '{}' deps: {} children ({} scanned) + {} from sub-tree"
        .format(str(node), len(children), len(scanned_deps), len(subtree_deps)))

return __FULL_DEPS[node]

Getting 700+ dependencies in about a second on a high performance machine. This should be called in a post-action, i.e. after the node is built. Probably will still work fine when called just before the node is built.

Upvotes: 0

Related Questions