Reputation: 1
I'm trying to create a dynamic copy activity which uses a GetMededata and a filter activity within data factory to do incremental loads.
The file path I'm trying to copy looks like :
2022/08/01
It goes from year folder - month folder - to days of the month folder and each day folder containing several files:
I need get metadata and filter activity to read the file, however the output directory on the filter activity foes not have the desired path.
The variable I've currently is set is dynamically expressed
@concat(variables('v_BlobSourceDirectory'), variables('v_Set_date'))
filter activity
@activity('Get Source').output.childItems
get metadata activity
@and(equals(item().type,'Folder'),endswith(item().name,variables('v_Set_Date')))
Upvotes: 0
Views: 4832
Reputation: 1
I have reproduced ADLSGen2 file path 2022/09/21 format added sample files.
Using Metadata activity all the child items has been pulled and given has input to foreach activity.
Added Dynamic values for month and days.
Every childitem is iterated through foreach activity.
for each activity output is given to copy.
**Result:**Every childitem is copied and stored in Dynamicoutput1
Upvotes: 0