Rebuild target if it was externally changed while the source remained the same

Question

I have a Flex source file and the result of converting it to C++ both stored in version control. (Pretty common practice. This allows compiling the program on machines that don't have Flex installed.)

In some situations, operations on version control may downgrade the target file while the source remains in the latest version. (This is intended.)

In such cases, I would like SCons to just build the target again, so it is up to date. However, it doesn't detect that the target file is outdated. It seems to only check if the source file has changed. Can I make SCons also check for changes in the target file while it's deciding if a rebuild is required?

---

You can test that behavior, using this one-line SConstruct:

Command('b', 'a', 'cp $SOURCE $TARGET')

If you have this SConstruct file and you run the following commands:

echo foo >a
scons -Q b
echo bar >b
scons -Q b

you get the this result:

+ echo foo >a
+ scons -Q b
cp a b
+ echo bar >b
+ scons -Q b
scons: `b' is up to date.

Answer 1

As demorgan has suggested in their answer, the solution is to write a new decider, which takes the target into consideration while calculating the result.

However, it is not necessary to write a custom code that stores the previous hash value. We can use the same mechanism that is used to store in the SCons database previous hash values of dependencies. If we use function my_file.get_csig() it will not only return the current hash for this file but also will store it, so it is accessible via my_file.get_stored_info().csig the next time.

(Remember to call get_csig() each time even if you don't need its value this time, to ensure the stored value is not outdated. Also remember that the object returned by get_stored_info() doesn't always have the field csig, just like the object prev_ni, so you need to check for that field in both cases.)

Here is an example of a custom decider that check hashes of the dependency and the target and cause a rebuild if any of them has changed:

def source_and_target_decider(dependency, target, prev_ni, repo_node=None):
    src_old_csig = prev_ni.csig if hasattr(prev_ni, 'csig') else None
    src_new_csig = dependency.get_csig()
    print(f'"{dependency}": {src_old_csig} -> {src_new_csig}')
    
    tgt_stored_info = target.get_stored_info()
    tgt_old_csig = tgt_stored_info.ninfo.csig if hasattr(tgt_stored_info.ninfo, 'csig') else None
    tgt_new_csig = target.get_csig()
    print(f'"{target}": {tgt_old_csig} -> {tgt_new_csig}')
    
    return src_new_csig != src_old_csig or tgt_new_csig != tgt_old_csig

Decider(source_and_target_decider)
Command('b', 'a', action=Copy("$TARGET", "$SOURCE"))

Answer 2

As specified in the documentation (https://scons.org/doc/production/HTML/scons-user.html#idp140211724034880), SCons will decide based on the input file. Only deleting the target will trigger a re-build since the target won't exist anymore.

For your case what you need is a Decider and this is documented under:

https://scons.org/doc/production/HTML/scons-user.html#idp140211709501040

I wrote a small example with a decider that will always decide that the target has to be rebuild:

def my_decider(dependency, target, prev_ni, repo_node=None):
    print("Executing my decider...")
    print("dep: %s " % dependency)
    print("target: %s" % target)
    print("prev_ni: %s" % prev_ni)
    print("repo node: %s" % repo_node)
    return True

Decider(my_decider)
Command('b', 'a', action=[
        Copy("$TARGET", "$SOURCE"),
    ]
)

The checksum for the target should be somewhere in the database of SCons and to retrieve it you can check the api:

https://scons.org/doc/latest/PDF/scons-api.pdf

If not get_timestamp() function should return the timestamp of the target. You can save this everytime in another file and then compare it in your decider.

EDIT

Below there is a possible implementation solution (yes it is with a file that you can ignore using the versioning system). This will work starting from the third build, since the decider is called twice with different targets....Also the decider for the files is left away intentionally, the code is already in the manual for the dependency part. I don't understand that SCons behavior at this moment and will try to get a solution for that later, since it requires too much time :)

import os

already_calculated = False

def get_name(target):
    return "." + str(target)

def store_hash(target):
    with open(get_name(target), "w+") as f:
        f.write(target.get_content_hash())
    f.close()

def get_hash(target):
    if not os.path.exists(get_name(target)):
        return None
    with open(get_name(target), "r") as f:
        return f.readline()

def my_decider(dependency, target, prev_ni, repo_node=None):
    global already_calculated
    if already_calculated:
        return
    already_calculated = True 
    if get_hash(target) == None:
        store_hash(target)
        return True
    old_hash = get_hash(target)
    new_hash = target.get_content_hash()
    if old_hash != new_hash:
        store_hash(target)
        return True
    return False

Decider(my_decider)
Command('b', 'a', action=[
        Copy("$TARGET", "$SOURCE"),
    ]
)