ADF Get Metadata only get files, not folders

Question

In my azure data factory pipeline, I am trying to get only the files in my dataset path but when it executes it brings in folders and files together. I have a metadata activity followed by ForEach which needs to only go through the files not the folder.

In my Field list of get metadata activity, I have Child Items listed . In my foreach activity, I have @activity('Get All Files in Folder').output.childItems is there. In my dataset file path I only have path to the folder listed.

Thanks for your help.

Answer 1

Another option is to add a Filter action after the GetMetadata:

In the Filter, you specify the item().Type as 'File' and any other pattern you wish:

Then the ForEach loops over the results of the Filter:

Answer 2

Most likely all the files have the same extension - for example csv. In this case you can go to 'Get Metadata' activity -> Settings -> Dataset -> fileName and filter only the *.csv

Answer 3

Add If-else activity inside the ForEach activity with below condition.

@equals(item().type,'File')

And add your activities you needed if it is file type.

Output:

Below is the files in directory.

SetVariable executed 5 time as there are only 5 files.

and