CODEWITHSUNDEEP
bashmakefile
earandap
earandap

Reputation: 1496

Validating file existence in a Makefile for executing a script

I am having a problem with validating the file existence in this script. The script fails when there is not a file in the ADDITION or the DELETIONS path.

How can I validate that there is a file in the ADDITION or DELETION path?

I'm doing my first steps with bash and Makefile so any help will be appreciated.

execute-s3-sync: $(foreach BUCKET,$(S3_BUCKETS),execute-$(BUCKET)-s3-sync) execute-large-files-s3-sync
execute-%-s3-sync: require-target-environment property-SOURCE_DATA_EXPORT_KEY property-TARGET_DATA_EXPORT_KEY
    export AWS_DEFAULT_REGION=$(TARGET_AWS_REGION)
    MANIFEST="$(if $(filter $*,large-files),large-files,$(call get-source-s3-bucket,$*)).gz"
    ADDITION="target/migrations/$(TARGET_DATA_EXPORT_KEY)/diff/$(SOURCE_DATA_EXPORT_KEY)/additions/$$MANIFEST"
    DELETION="target/migrations/$(TARGET_DATA_EXPORT_KEY)/diff/$(SOURCE_DATA_EXPORT_KEY)/deletions/$$MANIFEST"
    [[ -f "$$ADDITION" ]] && \
    pipenv run python scripts/large-files-copy.py \
        $(foreach BUCKET,$(S3_BUCKETS),--bucket-equivalences $(call get-source-s3-bucket,$(BUCKET)) $(call get-target-s3-bucket,$*)) \
        --manifest $$ADDITION \
        submit-to-batch \
            --job-queue "$(STACK_PREFIX)-batch-job-queue" \
            --job-definition "$(STACK_PREFIX)-batch-job-definition"
    [[ -f "$$DELETION" ]] && \
    pipenv run python scripts/large-files-copy.py \
        $(foreach BUCKET,$(S3_BUCKETS),--bucket-equivalences $(call get-source-s3-bucket,$(BUCKET)) $(call get-target-s3-bucket,$*)) \
        --manifest $$DELETION \
        local-run \
            --operation remove;

Upvotes: 2

Views: 122

Answers (1)

ensc
ensc

Reputation: 6994

you have to make this a huge, single bash command; make executes each line separately in the shell and variables are not available in later steps. To improve readability/maintainability, I would replace the test -f ... && xxx statements here by if blocks.

E.g.

execute-%-s3-sync: ...
    MANIFEST="$(if $(filter $*,large-files),large-files,$(call get-source-s3-bucket,$*)).gz" && \
    ADDITION="target/migrations/$(TARGET_DATA_EXPORT_KEY)/diff/$(SOURCE_DATA_EXPORT_KEY)/additions/$$MANIFEST" && \
    DELETION="target/migrations/$(TARGET_DATA_EXPORT_KEY)/diff/$(SOURCE_DATA_EXPORT_KEY)/deletions/$$MANIFEST" && \
    if test -f "$$ADDITION"; then \
       ...
    fi && \
    if test -f "$$DELETION"; then \
       ...

Regarding failing, you have to express things in a positive manner. E.g.

  • BAD:

    test -f $FILE && do_something    ## --> will fail when $FILE does not exists

  • GOOD:

    ! test -f $FILE || do_something  ## --> fails only, when do_something fails

  • GOOD:

    if test -f $FILE; then
   do_something
fi

Upvotes: 1

Related Questions