CODEWITHSUNDEEP
azureazure-sql-databaseazure-data-factoryazure-data-lake
Ashish Shelar
Ashish Shelar

Reputation: 63

Creating a multiple CSV files from a Single Table in Azure Data Factory

I have a Single SQL Table named an Employee Table. In Employee Table there are the following columns 1. ID, 2. Designation, 3. Salary, 4 Joining Date .. etc.

Now let say there are 200 records in this Employee Table, with 10 different designations of Employee. Now, I want to create a CSV file of this table using Azure Data Factory, based on(filter by) designations, i.e 10 CSV files(bcoz there are 10 different designations) should generate and store this in the storage account in Azure. Kindly guide me in this.

Upvotes: 0

Views: 1327

Answers (2)

Steve Johnson
Steve Johnson

Reputation: 8660

Another way: use Data Flow, set Key partition type and choose Designation as partition column at Source. Then choose Pattern as File name option and use employee[n].csv as file name.

Details:

1.set Key partition type and choose Designation as partition column at Source. enter image description here

2.choose Pattern as File name option and use employee[n].csv as file name enter image description here

enter image description here

Upvotes: 2

Nandan
Nandan

Reputation: 4925

You would need to follow the below flow chart:

LookUp Activity : Query : Select distinct designations from table For each activity Input : @activity('LookUp').output.value a) Copy activity i) Source : Dynamic Query Select * from t1 where designation=@item().designation This should generate separate files for each Designation as needed

For more details: How to export from sql server table to multiple csv files in Azure Data Factory

Upvotes: 3

Related Questions