CODEWITHSUNDEEP
pythonazureazure-data-factory
Joe
Joe

Reputation: 35

GET data from REST API using Azure Data Factory

I am trying to get data from Pendo through REST API using Azure Data Factory. The PYTHON code for this activity is 

import requests
url = "https://app.pendo.io/api/v1/aggregation"
data = "{\"response\":{\"mimeType\":\"application/json\"},\"request\":{\"pipeline\":[{\"source\":{\"guideEvents\":null,\"timeSeries\":{\"first\":\"1506977216000\",\"count\":-10,\"period\":\"dayRange\"}}}]}}"
headers = {
    'x-pendo-integration-key': "[add_your_pendo_int_key_here]",
    'content-type': "application/json"
}
response = requests.post(url, data = data, headers = headers)

How do I define the data part of the code through Azure Data Factory? I've put the content-type and x-pendo-integration-key in the additional headers.

Upvotes: 1

Views: 3970

Answers (2)

Joe
Joe

Reputation: 35

Problem solved. The data part of the PYTHON code(with out the \) is put in the Request Body of Copy Activity in Azure Data Factory and the Request Method is POST.

Upvotes: 1

Mohit Verma
Mohit Verma

Reputation: 5294

You can use HTTP Connector to retrieve data from HTTP endpoint. This HTTP connector is supported for the following activities:

  • Copy activity with supported source/sink matrix
  • Lookup activity You can copy data from an HTTP source to any supported sink data store. For a list of data stores that Copy Activity supports as sources and sinks, see Supported data stores and formats.

Note: To test an HTTP request for data retrieval before you configure the HTTP connector in Data Factory, learn about the API specification for header and body requirements. You can use tools like Postman or a web browser to validate.

https://learn.microsoft.com/en-us/azure/data-factory/connector-http

Here is a sample Custom activity for POST:

{
    "name": "HttpSourceDataInput",
    "properties": {
        "type": "HttpFile",
        "linkedServiceName": {
            "referenceName": "<HTTP linked service name>",
            "type": "LinkedServiceReference"
        },
        "typeProperties": {
            "relativeUrl": "<relative url>",
            "requestMethod": "Post",
            "requestBody": "<body for POST HTTP request>"
        }
    }
}

You can check ADF related samples here.Also find this for additional reference. Hope it helps.

Upvotes: 0

Related Questions