CODEWITHSUNDEEP
azure-data-factory
kerray
kerray

Reputation: 567

Azure Data Factory V2 - Compression type (enum) as parameter

I have a pipeline to copy blobs between storages.

I'd like to have Compression Type and Compression Level in the Sink Dataset (storage) set up as parameters.

When I edit the dataset as JSON, I'm able do use expressions and parameters, but there's no way to switch compression off (None), which would be represented as an empty object.

Is there a way to achieve this?

Thanks!

Upvotes: 1

Views: 668

Answers (1)

kerray
kerray

Reputation: 567

Ok, I finally figured it out - you have to directly edit JSON for the target linked storage service, and the expression I used is

            "compression": {
            "type": "Expression",
            "value": "@json(if(or(equals(dataset().TargetCompressionType, 'None'),equals(dataset().TargetCompressionType, '')),'{}',concat('{\"type\":\"', dataset().TargetCompressionType, '\", \"level\":\"', dataset().TargetCompressionLevel, '\"}')))"
        }

So when TargetCompressionType is set to empty string or "None", the result will be an empty object and compression won't be used, and when set to "ZipDeflate" or others, the files from source will be compressed.

Upvotes: 2

Related Questions