CODEWITHSUNDEEP
azureazure-data-factory
Mr. Blond
Mr. Blond

Reputation: 1167

Azure Data Factory File Name Option Pattern value picked from select activity output

We have multiple source .csv files in the Azure BLOB storage where each file has the same name as its "label" column value. We loop through each of these files via the ForEach loop which contains a single DataFlow:

enter image description here

The problem is that we were only able to use the "Default" value for the "File name option" field which sets the auto-generated GUID for the file name. We would like to use the "label" field's value within the "Pattern" field and change it a bit.

The "SelectColumns" step outputs the following mapping: enter image description here

Question: Is it possible to pass the "label" column's value to the "Pattern" field and modify it? For example, if the label's value is has_stuff the sink activity would output has_stuff_log.

If it's not possible, how would I pass the ForEach item().name to that field?

Upvotes: 0

Views: 436

Answers (1)

Rakesh Govindula
Rakesh Govindula

Reputation: 11514

As all of your values in the column label are same, add the '_log.csv' using derived column after the select transformation and use this column in the sink file name settings.

concat(label, '_log.csv')

enter image description here

Derived column results:

enter image description here

Now, use this column in the sink file name settings like below.

enter image description here

Result file after pipeline execution:

enter image description here

Upvotes: 1

Related Questions