CODEWITHSUNDEEP
azure-data-factory
Gregory Suvalian
Gregory Suvalian

Reputation: 3832

How do I use ADF copy activity with multiple rows in source?

I have source which is JSON array, sink is SQL server. When I use column mapping and see the code I can see mapping is done to first element of array so each run produces single record despite the fact that source has multiple records. How do I use copy activity to import ALL the rows?

 "enableStaging": false,
                    "translator": {
                        "type": "TabularTranslator",
                        "schemaMapping": {
                            "['@odata.context']": "BuyerFinancing",
                            "['@odata.nextLink']": "PropertyCondition",
                            "value[0].AssociationFee": "AssociationFee",
                            "value[0].AssociationFeeFrequency": "AssociationFeeFrequency",
                            "value[0].AssociationName": "AssociationName",

Upvotes: 0

Views: 2681

Answers (2)

Jambo
Jambo

Reputation: 61

Use * as the source field to indicate all elements in json format. For example, with json:

{
"results": [
     {"field1": "valuea", "field2": "valueb"},
     {"field1": "valuex", "field2": "valuey"}
    ]
}

and a database table with a column result to store the json. The mapping with results as the collection and * and the sub element will create two records with:

{"field1": "valuea", "field2": "valueb"}
{"field1": "valuex", "field2": "valuey"}

in the result field.

Copy Data Field Mapping

Upvotes: 1

Fang Liu
Fang Liu

Reputation: 2363

ADF support cross apply for json array. Please check the example in this doc. https://learn.microsoft.com/en-us/azure/data-factory/supported-file-formats-and-compression-codecs#jsonformat-example

For schema mapping: https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-schema-and-type-mapping#schema-mapping

Upvotes: -1

Related Questions