Azure Data Factory: How to retain the file name in the sink through a copy activity

Question

I'm trying to copy a file from SFTP to Blob Storage container. However, after the activity the filename in the Storage container seems completely random and different from that of the source file. How can I persist the file name?

I tried to follow this similar post , but in my case, i dont see the option of 'Dataset Properties' at all, neither for sink nor for source.

What am i missing?

Answer 1

but in my case, i dont see the option of 'Dataset Properties' at all, neither for sink nor for source

The dataset properties mean the dataset parameters. To see those, first you need to create dataset parameters in both source and sink datasets. As you want your file name to be dynamic, you need to give those parameters in the file name section of the datasets.

Create string type parameter without any default value like below.

Source dataset:

Now, give this in the dataset file name as @dataset().<parameter_name>. Give your source folder path.

To get the child file names from the source folder using Get meta data activity, the dataset path should be only till that folder. So, you cannot use the above dataset for your Get meta data activity as it has dynamic file path.

Create another dataset with path till your source folder and give that to your Get meta data activity at first.

Similarly, do the same for the sink dataset as well. Here, the parameter name is target_filename.

Now, the dataset properties will get appear for both source and sink in copy activity. Give the file name @item().name to these parameters.

In your sink, you have given @item().name expression to copy behavior. Remove it and give the same to the dataset parameter.

Now, the source file(s) will get copied to the target location with same file name.