CODEWITHSUNDEEP
azure-data-factory
ITHelpGuy
ITHelpGuy

Reputation: 1037

connect to REST endpoint using OAuth2

I am trying to explore different options to connect to a REST endpoint using Azure Data Factory. I have the below python code which does what I am looking for but not sure if Azure Data Factory offers something out of the box to connect to the api or a way to call a custom code.

Code:

import sys
import requests
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient
import json
import logging
import time     
logging.captureWarnings(True)
api_url = "https://webapi.com/api/v1/data"          
client_id = 'client'
client_secret = 'secret'     
client = BackendApplicationClient(client_id=client_id)
oauth = OAuth2Session(client=client)
token = oauth.fetch_token(token_url='https://webapi.com/connect/accesstoken', client_id=client_id, client_secret=client_secret)    
client = OAuth2Session(client_id, token=token)
response = client.get(api_url)
data = response.json()

When I look at the REST linked service I don't see many authentication options

enter image description here

Could you please point to me on what activities to use to make OAuth2 working in Azure Data Factory

Upvotes: 2

Views: 1696

Answers (1)

KarthikBhyresh-MT
KarthikBhyresh-MT

Reputation: 5034

You would have to use a WebActivity to call using POST method and get the authentication token before getting data from API.

Here is an example.

  1. First create an Web Activity.
  2. Select your URL that would do the authentication and get the token.
  3. Set Method to POST.
  4. Create header > Name: Content-Type Value: application/x-www-form-urlencoded
  5. Configure request body for HTTP request.

..

Format: grant_type=refresh_token&client_id={client_id}&client_secret=t0_0CxxxxxxxxOKyT8gWva3GPU0JxYhsQ-S1XfAIYaEYrpB&refresh_token={refresh_token}

Example: grant_type=refresh_token&client_id=HsdO3t5xxxxxxxxx0VBsbGYb&client_secret=t0_0CqU8oA5snIOKyT8gWxxxxxxxxxYhsQ-S1XfAIYaEYrpB&refresh_token={refresh_token

I have shown above for example, please replace with respective id and secret when you try.

enter image description here

As an output from this WebActivity, you would receive a JSON string. From which you can extract the access_token to further use in any request header from further activities (REST linked service) in the pipeline depending on your need.

You can get the access_token like below. I have assigned it to a variable for simplicity.

@activity('GetOauth2 token').output.access_token

enter image description here

Here is an example from official MS doc for Oauth authentication implementation for copying data.

Upvotes: 2

Related Questions