CODEWITHSUNDEEP
azure-data-factory
user989988
user989988

Reputation: 3736

What is the difference between ADF Pipeline and ADF Data flow?

What is the difference between ADF Pipeline and ADF Data flow? Why are sinks/sources supported in Pipeline and Data flow different? Is it possible to create a pipeline that reads data from source, filter, use joins and store data to a sink without a data flow? Please let me know.

Upvotes: 21

Views: 23444

Answers (1)

Mark Kromer MSFT
Mark Kromer MSFT

Reputation: 3838

Pipelines are for process orchestration. Data Flow is for data transformation.

In ADF, Data Flows are built on Spark using data that is in Azure (blob, adls, SQL, synapse, cosmosdb).

Connectors in pipelines are for copying data and job orchestration. There are 90+ connectors available there that stretch across on-prem and other clouds.

We are always incrementally adding more connectors to data flows.

Upvotes: 33

Related Questions