Azure Data Factory - Get Metadata inside for each activity

Question

folder structure:

raw

• test1
  • in.csv
  • out.csv
• test2
  • in.csv
  • out.csv
• test3
  • in.csv
  • out.csv
    
    

Here is what I want to do - use a get metadata activity to get a list of folders inside the raw folder. Then use a Foreach to go through the childitems of the get metadata activity and then inside the for each loop, use another get metadata activity that gets the metadata for every folder(all the test folders). This should work as new test folders are created (will have trigger to run pipeline), every folder will have the same structure and the same files inside but I need the get metadata to be able to work in the future for these folders that don't exist yet.
The issue I'm facing is setting the dataset for the Get Metadata that is inside the for loop since I can't set the dataset to the multiple test folders, some of which dont exist yet. I don't want to have to update the datasets everytime as I want the pipeline to run automatically with a trigger for when a new test folder is created. Thanks!

Answer 1

Please try this:

The screenshot of my pipeline:

The dataset of inside Get Metadata Activity:

If you aren't sure folder(test1,test2,test3) or csv file(in.csv,out.csv) exists, you can select 'Exists' in Get Metadata Activity like this:

Then you can use this value in output to confirm whether it exists,so you can do something else without error.

Hope this can help you.