CODEWITHSUNDEEP
azure-data-factory
kanderson
kanderson

Reputation: 177

Setting Default Value to 'None' for Parameter (Azure Data Factory)

I am currently trying to parametrize a dataset so that the compression type of a binary file can be set without creating a new dataset.

The issue that I am having is that I cannot seem to make the compression type default to 'None' while still having a parameter. I have tried typing in null, '', etc. but nothing seems to work. The pipeline will either not run, or it returns an error that it cannot be null, invalid type "" etc. Any advice would be appreciated.

Parameters in Files

I am trying to make the 'None' shown here the default value

Upvotes: 0

Views: 2517

Answers (2)

user21160098
user21160098

Reputation: 11

Not an obvious solution, but you can add a parameter named "CompressionType" to your dataset and then edit the dataset json to add this under "typeProperties":

"@if(equals(dataset().CompressionType,'None'),'no_compression','compression')": {
    "type": "@dataset().CompressionType"
}

Upvotes: 0

KarthikBhyresh-MT
KarthikBhyresh-MT

Reputation: 5044

--Update

It so appears this is something to do with how Binary dataset is designed.

Going through the Binary Dataset code I can see the below difference in usage of compression method/code block.

It indeed leads to an error for default value set to None.

enter image description here

And when compression type is set to None

enter image description here

enter image description here

enter image description here

You can reach out to support for official response or log an issue here or share an idea here

Also, it works fine when using the Default None option from the dropdown for compression properties in case of Binary dataset.

Checkout..

enter image description here

However...

I have tried this at both source and sink with csv files - Source.csv and Source.csv.gz both ways

You can set the parameter default value to None. And it works just fine.

enter image description here

Same is expected for Binary format Dataset properties but in vain 😕

Upvotes: 2

Related Questions