CODEWITHSUNDEEP
azureazure-data-factorydataflowtask
Dmitriy Ryabin
Dmitriy Ryabin

Reputation: 468

Azure data factory - Data flow External call - specify response body schema

I .. simply do not understand what I should do with this. I have a REST call that gets the list of Projects. I set up that as a source to the data flow - having a Rest Linked service feed the data.

Data Flow

The JSON from the REST call is list of Projects objects Something like ...

{
"_id": "5a04b1a0",
"name": "Some name - as string",
"number": "179",
"value": null,
"state": "PUBLISHED",
"Sealed": 0,
"ndaRequired": 0,
"public": 1
}

and there can be hundreds of those. After the list is obtained, I need to go through each one and make additiona API call to https:///projects/<{_id}> where _id is a project ID (from above) This call will give me individual project with ADDITIONAL fields - nested "Packages" array

{
"_id": "5a04b1a0",
"name": "Some name - as string",
"number": "179",
"value": null,
"state": "PUBLISHED",
"Sealed": 0,
"ndaRequired": 0,
"public": 1,
"Packages": [
    {
        "_id": "59a0471db3",
        "projectId": "5a04b1a0",
        "name": "some",
        "number": "9250",
        "keywords": [
            "keyWord"
        ],
        "state": "PUBLISHED",
        "dateStart": null
    },
    {
        "_id": "934234kkd93",
        "projectId": "5a04b1a0",
        "name": "some other",
        "number": "24349374",
        "keywords": [
            "keyWord, keyword, keyword"
        ],
        "state": "UNKNOWN",
        "dateStart": null
    }
}

I do not know how (The syntax??) to specify the response schema for each individual "GetPackages" external call. please help. In the "Data Preview" of the FlattenProjects stage, I get the correct list for Projects, and after "Derived Column" I get the additional "ProjectID" as a url to be added. If I test the complete individual URL in postman - it works, question is NOT about correctness of the URL call. what do I put in the "body" and/or "Type", so to get the packages added. ? I can't find any documentation. I can and know how to use FLatten transformation if I need to flatten it afterwards

External call What do I need to put into the "Type" in the picture, so I can get the

Upvotes: 0

Views: 1072

Answers (1)

Aswin
Aswin

Reputation: 7126

  • Click on the import projection in Output tab of external call transformation.
  • ADF will automatically detect the schema of the API call and Body type expression will be auto filled.

enter image description here

  • You can also manually specify the body data structure in type.

syntax: ( column1 as datatype, column2 as datatype.....)

Replace column1, column2 of above expression with actual column names and give the respective datatypes there.

sample expression: (updateTime as string, updated as string, validTimes as string)

Upvotes: 0

Related Questions