CODEWITHSUNDEEP
azure-cosmosdbazure-cosmosdb-sqlapi
user989988
user989988

Reputation: 3736

Issue with join in Azure Data Factory

I have created a simple ADF pipeline that has two sources (S1, S2) and stores data from these sources into Azure Cosmos DB sink using left outer join (condition: s1.abc = S2.abc). After running this pipeline, I can see all columns from S1 and none of the columns from S2. Why is that? Please help me understand.

Upvotes: 0

Views: 2146

Answers (2)

Mehmet Avni CELIK
Mehmet Avni CELIK

Reputation: 572

you should be careful about the schema of the dataset. when you create it "import schema" must be selected correctly. your problem might be about that as well. here how i solved a similar problem:

i was processing some csv files and did select the schema as none. because of this, the key in join conditions did not appeared there. after i change it to "from connection/store", join flow executed correctly and the data were visible in preview tab.

enter image description here

Upvotes: 1

Jay Gong
Jay Gong

Reputation: 23782

I can see all columns from S1 and none of the columns from S2

Since you mentioned left outer join in your question, so i think you are using Data Flow Activity to transfer the data. I tested on my side an it works for me.

Firstly,please check the statement of left outer join in the official document:

enter image description here

Then please refer to my sample test:

I have 2 csv files as below:

enter image description here

enter image description here

My data flow activity as below, the B is the join key:

enter image description here

Output in cosmos db,the row from left stream has no match,so the output from right stream is NULL:

enter image description here

Upvotes: 2

Related Questions