CODEWITHSUNDEEP
azureazure-data-factory
usr_lal123
usr_lal123

Reputation: 838

Remove specific columns using Azure Data Factory

I have set of files in ADLS. Number of columns varies in each file.

File 1 will have the columns : Row_Number, Col_A, Col_B, null

File 2 will have the columns : Row_Number, Col_1, Col_2, Col_3 , null

File 3 will have the columns : Row_Number, Col_A1, Col_A2, null

There will be more than 50 files in my ADLS. I would need to remove first and last column in each file or to be specific , I need to remove columns which has columns names as Row_Number and null.

Can any help me what should be my expressions in Derived column / Select.

Thanks for your help.

Upvotes: 2

Views: 6592

Answers (1)

Mark Kromer MSFT
Mark Kromer MSFT

Reputation: 3838

I would use a Select rule with this for the matching condition:

name != 'Row_Number' && name != 'null' && left(name,2) != '_c'

and this for the output column name: $$

enter image description here

Upvotes: 4

Related Questions