Azure Data Factory Copy activity only copying table structure and not actual data

Question

I'm running an ADF pipeline that looks like: Get Metadata -> For Each Loop -> Copy Activity (inside the For Each).

The files I am trying to copy are all text files and I am trying to copy the headers as columns and the actual data inside the files over to my SQL server database (which I'm creating before I actually run the activity).

The problem is that it is successfully copying the intended table structure based on the .txt file, but it is not copying any of the data over.

I was expecting the activity to copy over the table structure and the data itself, but it is not copying the data over.

I'm not able to share actual data, but the structure is as so for one of the files:

id,line1,line2,city,postal,updated_at,created_at,updated_by_id,created_by_id,country_subdivision,country_code

Answer 1

I tried to reproduce the same and copied all the files into the database with table structure and data.

• First, I took Get metadata activity to get the list of files with Field list as Child Items. Dataset of Get metadata activity:
• Then take a for each activity and pass this output of get metadata activity to foreach activity
• Under foreach activity take copy activity and add your source and sink with parameters.

Create source dataset parameter. Add parameter to connection. In source dataset parameter add value for dataset parameter as item().name.

Create sink dataset parameter. Add parameter to connection. In source dataset parameter add value for dataset parameter as item().name.

Output:

It will copy all the files into the database with table structure and data.