WildJester
WildJester

Reputation: 1

Azure Data Factory Dynamic Folder Path

I'm trying to create a dynamic copy activity which uses a GetMededata and a filter activity within data factory to do incremental loads.

The file path I'm trying to copy looks like : 2022/08/01
It goes from year folder - month folder - to days of the month folder and each day folder containing several files:

I need get metadata and filter activity to read the file, however the output directory on the filter activity foes not have the desired path.

enter image description here

The variable I've currently is set is dynamically expressed
@concat(variables('v_BlobSourceDirectory'), variables('v_Set_date'))

filter activity
@activity('Get Source').output.childItems

get metadata activity
@and(equals(item().type,'Folder'),endswith(item().name,variables('v_Set_Date')))

Upvotes: 0

Views: 4832

Answers (1)

Rapally Anusha
Rapally Anusha

Reputation: 1

I have reproduced ADLSGen2 file path 2022/09/21 format added sample files.

enter image description here

Using Metadata activity all the child items has been pulled and given has input to foreach activity.

enter image description here Added Dynamic values for month and days.

enter image description here Every childitem is iterated through foreach activity.

enter image description here for each activity output is given to copy.

enter image description here enter image description here

enter image description here

**Result:**Every childitem is copied and stored in Dynamicoutput1

enter image description here

Upvotes: 0

Related Questions