CODEWITHSUNDEEP
azurecsvpipelinefactory
Para Jaco
Para Jaco

Reputation: 211

Azure Data Factory pipeline to process each line in CSV separately and conditionally copy data into Dynamics CRM

I need some help with the following issue. I have a bunch of CSV files which, in my pipeline, I read from Azure BLOB storage. Now each CSV file contains a list of records, and all CSV files use the exact same data format/columns. I must decide per CSV line/record that is processed, if I want to create a Dynamics CE contact for it or not, based on some logic. My current pipeline first gets all the CSV files/records into one dataset, and then uses a For-Each loop to process each line/record separately. Inside the For-Each I have a conditional check to see if the current line must be used or not. If the data should be used, then how do I create a contact in Dynamics using the data from this record? It seems the Copy-Data activity only works with a dataset, while inside my For-Each I only have access to the current @item() ???

Upvotes: 0

Views: 201

Answers (1)

KamalDeep
KamalDeep

Reputation: 834

As I understood from the question, you have bunch of CSV files and you'r intention is to copy data from these files to Dynamics CRM.

  1. If you are having PowerApps also, then ADFS is not the recommended approach. You can use DataFlows in the PowerApps to achieve your task. This has added advantage that it gives you the power query by default so that you can easily do the transformations. Please refer this document for your reference.
  2. If you are not comfortable with Dataflow or for some reason you require ADFS only, then in ADFS there is a option of PowerQuery in the pipeline, you can use that and do the transformations as you like it.

Upvotes: 0

Related Questions