CODEWITHSUNDEEP
azureazure-data-factoryazure-data-lakeazure-data-lake-gen2data-lake
sai m
sai m

Reputation: 129

Column names are incorrectly Mapped

I was trying to pull/load data from on-prem data lake to azure data lake using Azure Data Factory.

I was just giving query to pull all the columns. My Sink is Azure Data Lake Gen2.

enter image description here

But my Column names are coming wrong in source and sink.
My columns name in on-prem data lake are like user_id, lst_nm, etc. But in Azure it is like user_tbl.user_id, user_tbl.lst_nm , etc Here user_tbl is my table name.
I don't want table name getting added to columns.

enter image description here

Upvotes: 0

Views: 723

Answers (2)

Utkarsh Pal
Utkarsh Pal

Reputation: 4552

Instead of using Copy activity, use Data flow transformation which allows you to change the Column name at destination dynamically.

Or you can also use Move and transform activity which also allows you to change column name. Refer official tutorial: Dynamically set column names in data flows

Also check ADF Mapping Data Flows: Create rules to modify column names

Upvotes: 0

Pratik Somaiya
Pratik Somaiya

Reputation: 743

Azure won't add the table name itself to the columns, can you check the output of select query that you are sending to source using preview data in ADF, that will show you the actual column names ADF is getting from source and if it doesn't have the table name prefixed then please check if your ADLS Gen 2 destination folder already have any file, if yes then remove the file and try running the pipeline again

Upvotes: 0

Related Questions