CODEWITHSUNDEEP
sqldatabaseazureazure-data-factorybulkinsert
Shameel Faraz
Shameel Faraz

Reputation: 78

Bulk copy multiple csv files from Blob Container to Azure SQL Database

Environment:

MS Azure: Blob Container, multiple csv files saved in a folder. This is my source. Azure Sql Database. This is my target

Goal: Use Azure Data Factory and build a pipeline to "copy" all files from the container and store them in their respective tables in the Azure Sql database by automatically creating those tables.

How do I do that? I tried following this but I just end up having tables incorrectly created in the database, where table is created with a single column having same name as the table name.

I believe I followed the instructions from that link pretty must as they are.

Upvotes: 1

Views: 831

Answers (1)

Joseph Xu
Joseph Xu

Reputation: 6043

My CSV file is as follows, one column contains the table name.
enter image description here

The previous steps will not be repeated,it is the same as the link.

  1. At Step3 inside the Foreach activity, we should add a Lookup activity to query the table name from the source dataset.
    enter image description here

  2. We can declare a String type variable tableName pervious, then set the value via expression @activity('Lookup1').output.firstRow.tableName. enter image description here

  3. At sink setting of the Copy activity, we can key in @variables('tableName'). enter image description here

  4. ADF will auto create the table for us. enter image description here

  5. The debug result is as follows:
    enter image description here

Upvotes: 2

Related Questions