CODEWITHSUNDEEP
azureazure-data-factoryazure-synapse
gio.s
gio.s

Reputation: 83

Azure synapse get metadata

I am trying to get a list of all files in a folder with get metadata activity. To pass this list to the for-each activity, which in turn executes a notebook.

I have a binary dataset and field list is set to child items.

Pipeline crashes every time with the error:

{
    "errorCode": "2011",
    "message": "Blob operation Failed. ContainerName: tmp, path: /tmp/folder/folder1/.",
    "failureType": "UserError",
    "target": "Get Metadata",
    "details": []
}

The files are in 'folder/folder1'.

It's not my first time working with Get Metadata activity and so far it has always worked (in ADF). But I do it first time in Synapse, are there differences? Do you have any ideas what this could be or how I can solve the problem?

Upvotes: 1

Views: 2374

Answers (1)

NiharikaMoola
NiharikaMoola

Reputation: 5074

Usage of Get Metadata activity to retrieve the metadata of any data is the same in the Azure data factory and Azure Synapse pipeline.

  1. Create a binary dataset with a dataset parameter for filename.

enter image description here

  1. Connect the binary dataset to the Get metadata activity.
  • Pass '*' to the filename parameter value.
  • Select child items under Field list to get the list of files/subfolders from the folder.

enter image description here

  1. The output gives the list of files from the folder.

enter image description here

Upvotes: 1

Related Questions