How do i authenticate to and get catalog from Azure Data Catalog using Rest API in Python

Question

I would like to get the name of the catalog from Azure Data Catalog using API. When I tried using the following command to get catalog from Azure Data Catalog

requests.get("https://management.azure.com/subscriptions/{id}/resourceGroups/{group_name}/providers/Microsoft.DataCatalog/catalogs/{catalogname}")

as mentioned in the link https://learn.microsoft.com/en-us/rest/api/datacatalog/data-catalog-data-catalog

It throws the following error

Response [400]

Looks like I have to authenticate first. How do I authenticate prior to getting catalog?

Answer 1

Adding the new answer in python

for getting the auth context in python you could do the following

here is the settings for the parameters we need it while calling graph api.

RESOURCE = "https://graph.microsoft.com"  # Add the resource you want the access token for
TENANT = "Your tenant"  # Enter tenant name, e.g. contoso.onmicrosoft.com
AUTHORITY_HOST_URL = "https://login.microsoftonline.com"
CLIENT_ID = "Your client id "  # copy the Application ID of your app from your Azure portal
CLIENT_SECRET = "Your client secret"  # copy the value of key you generated when setting up the application

# These settings are for the Microsoft Graph API Call
API_VERSION = 'v1.0'

here is the code for logging in

AUTHORITY_URL = config.AUTHORITY_HOST_URL + '/' + config.TENANT
REDIRECT_URI = 'http://localhost:{}/getAToken'.format(PORT)
TEMPLATE_AUTHZ_URL = ('https://login.microsoftonline.com/{}/oauth2/authorize?' +
                      'response_type=code&client_id={}&redirect_uri={}&' +
                      'state={}&resource={}')
                   

 def login():
    auth_state = str(uuid.uuid4())
    flask.session['state'] = auth_state
    authorization_url = TEMPLATE_AUTHZ_URL.format(
        config.TENANT,
        config.CLIENT_ID,
        REDIRECT_URI,
        auth_state,
        config.RESOURCE)
    resp = flask.Response(status=307)
    resp.headers['location'] = authorization_url
    return resp

here is how you can retrieve the token

 auth_context = adal.AuthenticationContext(AUTHORITY_URL)
    token_response = auth_context.acquire_token_with_authorization_code(code, REDIRECT_URI, config.RESOURCE,
                                                                        config.CLIENT_ID, config.CLIENT_SECRET)

and then you can create a endpoint for your azure data catalog api. Here is the http header for the same-

http_headers = {'Authorization': 'Bearer ' + token_response['accessToken'],
                    'User-Agent': 'adal-python-sample',
                    'Accept': 'application/json',
                    'Content-Type': 'application/json',
                    'client-request-id': str(uuid.uuid4())}

and then finally you can call the api. Here endpoint being the data catalog API URL.

data = requests.get(endpoint, headers=http_headers, stream=False).json()

Hope it helps.

Answer 2

To call a Data Catalog REST operation, create an instance of AuthenticationContext and call AcquireToken. AuthenticationContext is part of the Active Directory Authentication Library NuGet package. To install the Active Directory Authentication Library NuGet package in Visual Studio,run

 "Install-Package Microsoft.IdentityModel.Clients.ActiveDirectory" 

from the NuGet Package Manager Console.

Here is the code to get the token for the same.

static async Task<AuthenticationResult> AccessToken()
        {
            if (authResult == null)
            {
                //Resource Uri for Data Catalog API
                string resourceUri = "https://api.azuredatacatalog.com";

                //To learn how to register a client app and get a Client ID, see https://msdn.microsoft.com/en-us/library/azure/mt403303.aspx#clientID   
                string clientId = clientIDFromAzureAppRegistration;

                //A redirect uri gives AAD more details about the specific application that it will authenticate.
                //Since a client app does not have an external service to redirect to, this Uri is the standard placeholder for a client app.
                string redirectUri = "https://login.live.com/oauth20_desktop.srf";

                // Create an instance of AuthenticationContext to acquire an Azure access token
                // OAuth2 authority Uri
                string authorityUri = "https://login.windows.net/common/oauth2/authorize";
                AuthenticationContext authContext = new AuthenticationContext(authorityUri);

                // Call AcquireToken to get an Azure token from Azure Active Directory token issuance endpoint
                //  AcquireToken takes a Client Id that Azure AD creates when you register your client app.
                authResult = await authContext.AcquireTokenAsync(resourceUri, clientId, new Uri(redirectUri), new PlatformParameters(PromptBehavior.Always));
            }

            return authResult;
        }

Here is the sample code to get the data asset base on id

// The Get Data Asset operation retrieves data asset by Id
        static JObject GetDataAsset(string assetUrl)
        {
            string fullUri = string.Format("{0}?api-version=2016-03-30", assetUrl);

            //Create a GET WebRequest as a Json content type
            HttpWebRequest request = WebRequest.Create(fullUri) as HttpWebRequest;
            request.KeepAlive = true;
            request.Method = "GET";
            request.Accept = "application/json;adc.metadata=full";

            try
            {
                var response = SetRequestAndGetResponse(request);
                using (var reader = new StreamReader(response.GetResponseStream()))
                {
                    var itemPayload = reader.ReadToEnd();
                    Console.WriteLine(itemPayload);
                    return JObject.Parse(itemPayload);
                }
            }
            catch (WebException ex)
            {
                Console.WriteLine(ex.Message);
                Console.WriteLine(ex.Status);
                if (ex.Response != null)
                {
                    // can use ex.Response.Status, .StatusDescription
                    if (ex.Response.ContentLength != 0)
                    {
                        using (var stream = ex.Response.GetResponseStream())
                        {
                            using (var reader = new StreamReader(stream))
                            {
                                Console.WriteLine(reader.ReadToEnd());
                            }
                        }
                    }
                }
            }

            return null;
        }

Here is how you can set the request, token and get the response.

  static HttpWebResponse SetRequestAndGetResponse(HttpWebRequest request, string payload = null)
        {
            while (true)
            {
                //To authorize the operation call, you need an access token which is part of the Authorization header
                request.Headers.Add("Authorization", AccessToken().Result.CreateAuthorizationHeader());
                //Set to false to be able to intercept redirects
                request.AllowAutoRedirect = false;

                if (!string.IsNullOrEmpty(payload))
                {
                    byte[] byteArray = Encoding.UTF8.GetBytes(payload);
                    request.ContentLength = byteArray.Length;
                    request.ContentType = "application/json";
                    //Write JSON byte[] into a Stream
                    request.GetRequestStream().Write(byteArray, 0, byteArray.Length);
                }
                else
                {
                    request.ContentLength = 0;
                }

                HttpWebResponse response = request.GetResponse() as HttpWebResponse;

                // Requests to **Azure Data Catalog (ADC)** may return an HTTP 302 response to indicate
                // redirection to a different endpoint. In response to a 302, the caller must re-issue
                // the request to the URL specified by the Location response header. 
                if (response.StatusCode == HttpStatusCode.Redirect)
                {
                    string redirectedUrl = response.Headers["Location"];
                    HttpWebRequest nextRequest = WebRequest.Create(redirectedUrl) as HttpWebRequest;
                    nextRequest.Method = request.Method;
                    request = nextRequest;
                }
                else
                {
                    return response;
                }
            }
        }

Basically you need to get the bearer token and pass it as a request parameter to get the catalog using azure data catalog api.

for further code sample, please browse below code repository.

https://github.com/Azure-Samples/data-catalog-dotnet-get-started

Hope it helps.