CODEWITHSUNDEEP
jsonazure-data-factory
Matt Owen
Matt Owen

Reputation: 37

Data Factory: API call returning JSON format wont go to JSON file

In data factory I'm using a Copy Data activity where the Source is a REST API, that's returning data in JSON format, with a Sink thats a JSON file type in ADLS Storage.

The problem I'm having is that only the first record in the "Domestic Title Nos" array appears in the JSON file. The remaining records, however many, are not to be seen.

Also, when I "Preview" my data in the Source page, it shows all the nested data in the array.

API Call Returns this JSON data

Mapping in my Copy Data activity

File contents after sink to JSON file type

Upvotes: 0

Views: 192

Answers (2)

Matt Owen
Matt Owen

Reputation: 37

In the end it worked out to be easier to download the entire JSON document using this mapping and then used "Flatten" in a Data Flow to extract the parts that I needed.

Upvotes: 0

Abhishek Khandave
Abhishek Khandave

Reputation: 3228

As shown in below screenshot, you have used items as Collection reference.

You need to use “Domestic Title Nos” as Collection reference.

enter image description here

Note: For records where the array marked as collection reference is empty and the check box is selected, the entire record is skipped.

Refer - Schema and data type mapping in copy activity

Upvotes: 0

Related Questions