How to Get Row Count of the Result from Exists Activity in Azure Data Factory Data Flow?

Question

I am currently working on a data flow in Azure Data Factory where I need to check if the contents of a CSV file match with the contents of an SQL table. For this purpose, I am using the Exists activity.

While I am able to see the results of the Exists activity in the Data Preview section, I am struggling to find a way to get the total number of rows in the result set.

Here is what I have done so far:

• Created a data flow.
• Used the Exists activity to compare the CSV file (Blob Storage) with the Azure SQL table.
• Previewed the data using the Data Preview section, which shows the matching rows.

However, I need to get the count of these rows programmatically so I can use it in further activities within the pipeline.

My questions are:

• How can I get the row count of the result from the Exists activity?
• Is there a built-in function or expression in Azure Data Factory that can help me achieve this?
• Do I need to add additional activities or transformations to capture the row count, and if so, which ones?

Any guidance or examples would be greatly appreciated!

Additional Details:

• I am using Azure Data Factory version 2.
• The CSV file is stored in Azure Blob Storage.
• The SQL table is hosted in Azure SQL Database.

Thank you!

Answer 1

If you are using exists transformation before your SQL sink in the dataflow, you can get the record count like below.

The dataflow activity output JSON will contain the rowsWritten count to the sink.

To get this count in the ADF pipeline, use the below expression after dataflow activity.

@activity('Data flow1').output.runStatus.metrics.<sink name>.rowsWritten

Result: