nam
nam

Reputation: 23809

Azure mapping data flow - how to arrange transformations

I created an Azure Mapping Data Flow with Aggregate transformation that shows in the first row of the diagram below. Then I added a new branch to the Employee source, and then new data source Department. But, as shown in figure 2 below, when I try to join branched data source Employee with the department data source, I was expecting the join transformation to appear in between branched data source Employee and the department. But instead, the join appears next to the first data source Employee on top, and the Aggregate transform along with the Sink data source move down to second row (as shown in figure 3 below). Question: Why this is happening and how can I move the Join transformation between the branched data source (Employee) and the Department?

Employee Source with its Branch (below it) and Aggregate and Sink next to it:

enter image description here

Adding Join transformation between Branched Employee source and Department (hoping the Join will appear between Branched Employee source and Department):

enter image description here

But, instead following happened: Why? Note: I can still join the Employee and Department by it will look ugly that left part of the join is on top, and the right part (Department) of the join is below the second row.

enter image description here

Expected Graph [taken from another document on Data Factory]: Something like this I wanted

enter image description here

Upvotes: 0

Views: 489

Answers (1)

Mark Kromer MSFT
Mark Kromer MSFT

Reputation: 3838

Mapping Data Flows is not a free-flow diagram. It is a construction graph that will automatically adjust the node positions, connecting lines, and reference nodes for you to optimize space in the graph. Your diagrams above are expressing the exact same semantic to ADF. The only difference is that the UI is moving the Join node to the top stream. If you look at the script behind that data flow, you'll note that ADF sees the original Employee source and the new branch of Employee as the same object. If you'd like to influence the placement of the streams in your graph, open the script behind your flow and you can swap the lines something like this:

Employee aggregate(newfield = sum(1)) ~> Aggregate1 Employee, Department join(surrogatekey == movie, joinType:'inner', broadcast: 'auto')~> Join1

... change it to this with Join on top ...

Employee, Department join(surrogatekey == movie, joinType:'inner', broadcast: 'auto')~> Join1 Employee aggregate(newfield = sum(1)) ~> Aggregate1

That will swamp the order of the streams in your flow and should give you what you are looking for.

Upvotes: 1

Related Questions