CODEWITHSUNDEEP
azureazure-data-factory
robert_553z8
robert_553z8

Reputation: 191

data factory data flows - path does not resolve to any file(s)

trying to set up a data flow to de-duplicate data but there seems to be a simple error that I cannot fix, the data set is set up like so and the preview works correctly - updated file path and imported schema enter image description here

the data flow is set up like this enter image description here

however I get this error when trying Data preview - updated -> got the columns from importing the schema in the dataset

enter image description here

I have tried editing wildcard paths in source options and so far that has not worked i tried

daily/*csv and *.csv

the structure of the blob account looks like so: enter image description here

source options from the answer given: enter image description here

still the error: Path does not resolve to any file(s). Please make sure the file/folder exists and is not hidden. At the same time, ensure special character is not included in file/folder name, for example, name start with _

each directory is from an azure export and creates its own monthly folder - this works on data factory but each csv has month to date cost so I was trying to use data flows to only take new data instead of all of the data with duplicates enter image description here

Upvotes: 3

Views: 3594

Answers (1)

NiharikaMoola
NiharikaMoola

Reputation: 5074

You can use the wildcard path below to get the files of the required type.

Input folder path:

enter image description here

Azure data flow:

  1. Source dataset

enter image description here

  1. Source transformation: In source options provide the wildcard path to get the files of the required extension type. I have also included columns to store filenames to verify the data from all the files.

Wildcard paths : daily/*.csv

enter image description here

  1. Import projections and preview data.

enter image description here

Upvotes: 4

Related Questions