CODEWITHSUNDEEP
pythondoit
layoutanalysis
layoutanalysis

Reputation: 3

Unneeded re-run of doit task pipeline

I am using doit to automate my data analysis pipeline as described in https://blogs.aalto.fi/marijn/2016/02/25/doit-a-python-alternative-to-make/. Unfortunately, doit always wants to re-run the entire pipeline from step1 although the file_deps for the later steps are already satisfied. How can I find out what caused doit to re-run the entire pipeline? E.g. a misconfigured dependency, changes in dodo.py?

Upvotes: 0

Views: 296

Answers (1)

Dan Gilbert
Dan Gilbert

Reputation: 16

Sample code would be helpful. With out knowing exactly whats going on it maybe hard to help. But in hopes of leading you in the right direction, here are some pointers;

doit has strace functionality, which has some caveats but may be of help, http://pydoit.org/cmd_other.html?highlight=strace#strace .

By default, doit will not rerun tasks if your dodo.py file has changed. I would check your code to see if you have added your dodo.py file as a file_deps someplace; http://pydoit.org/faq.html#dodo-py-file-itself-should-be-a-file-dep-for-all-tasks

If you are still having issues, check out the users group and post there. Edward seems to be quick to respond.

Upvotes: 0

Related Questions