CODEWITHSUNDEEP
azureazure-data-factorycortana-intelligence
HHH
HHH

Reputation: 6485

How to delete a folder in an azure data factory pipeline?

When running MapReduce jobs, the output folder should not exist otherwise the job fails. So when defining an azure data factory how can I define an activity to delete yhe existing output folder? Oozie has a preprocessing activity that can be used to delete the a folder but I don't kbow how to do it in data factory?

Upvotes: 3

Views: 3634

Answers (1)

Nava Vaisman Levy
Nava Vaisman Levy

Reputation: 514

You can add a custom activity prior to the MapReduce activity that will delete the folder using a programmatic approach by writing an executable and invoking it from the pipeline. Use this tutorial as reference and examples . Basically you will need to upload the executable to a blob storage and Data Factory will execute it.

Upvotes: 1

Related Questions