Add file name as column in data factory pipeline destination

Question

I am new to DF. i am loading bunch of csv files into a table and i would like to capture the name of the csv file as a new column in the destination table.

Can someone please help how i can achieve this ? thanks in advance

Answer 1

If you use a Mapping data flow, there is an option under source settings to hold the File name being used. And later in it can be mapped to column in Sink.

Answer 2

1. A post said the could use data bricks to handle this. Data Factory - append fields to JSON sink

2. Another post said they are using USQL to hanlde this. use adf pipeline parameters as source to sink columns while mapping

3. For stored procedure, please reference this post. Azure Data Factory mapping 2 columns in one column

Answer 3

If your destination is azure table storage, you could put your filename into partition key column. Otherwise, I think there is no native way to do this with ADF. You may need custom activity or stored procedure.