CODEWITHSUNDEEP
mongodbazureazure-data-factoryazure-data-lake-gen2
Matzka
Matzka

Reputation: 135

Data Factory Pipeline failed with various errors

I set up an Azure Data Factory with a MongoDB as a source and a Delta Lake Storage Gen2. All connections (to the source and the target destination) were successfully checked. But the execution of the pipeline failed with the following errors:

*ADLS Gen2 operation failed for: Operation returned an invalid status code 'Forbidden'. Account: 'xxxx'. FileSystem: 'marketing'. Path: 'output/users.txt'. ErrorCode: 'AuthorizationPermissionMismatch'

"errorCode": "2200", "message": "Failure happened on 'Source' side. ErrorCode=UserErrorTypeInSchemaTableNotSupported,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to get the type from schema table. This could be caused by missing Sql Server System CLR

"errorCode": "2200", "message": "Failure happened on 'Sink' side. ErrorCode=UserErrorSchemaMappingCannotInferSinkColumnType,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Data type of column 'xx' can't be inferred from 1st row of data, please specify its data type in mappings of copy activity or structure of DataSet *

Any ideas how I can fix those errors?

Upvotes: 0

Views: 2016

Answers (1)

Himanshu Kumar Sinha
Himanshu Kumar Sinha

Reputation: 1786

The error "*ADLS Gen2 operation failed for: Operation returned an invalid status code 'Forbidden'. Account: 'xxxx'. FileSystem: 'marketing'. Path: 'output/users.txt'. ErrorCode: 'AuthorizationPermissionMismatch'"

points to the fact the pipeline does not have the required permission on the ADLS Gen2 . Please go through this doc : https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-access-control.

Please beware that in GEN2 the permissions are more granular and so when to check connections on the ADF UI that mey not catch that .

Let me know how it goes .

Thanks Himanshu

Upvotes: 1

Related Questions