How to only Debug 1 Activity in a Synapse Analytics Pipeline that has multiple activities

Question

I'm working in Synapse Studio and created an Azure Data Lake Storage Gen2 with various folders and creating parquet files through Pipeline Activities My Activities are to pull in source SQL Server Views into Parquet files that I can then export to Dynamics when processed I've created various Pipelines for different categories In one of my Pipelines I have 5 different Activities for 5 different Views I'd like to be able to Debug only 1 of the 5 Activities at a time without refreshing all the files. Is there a way to do this Or is it best practice to have 1 Activity per Pipeline? Cheers!

Answer 1

There is debug option in Azure Synapse and Azure Data factory to put a break point in the activity that you want to debug and then debug the ETL. only the once with debug point will run.

Answer 2

Another workaround could be to make the activity that you want to debug as the first activity in the pipeline. Then mark it for "Debug until". Refer the sample pipeline image

I think this would be easier than cloning the pipeline and deleting unnecessary activities.

Answer 3

Currently, the feature to run a single/specific activity in a pipeline that has multiple activities is not supported in Azure Synapse Analytics or the Azure Data factory.

If possible, you can clone the pipeline and delete other activities to debug.

You can raise a feature suggestion from the azure synapse analytics under feedback