azureoauth-2.0azure-active-directoryazure-databricks

Reputation: 6933

Error 403 User not authorized when trying to access Azure Databricks API through Active Directory

I have been following the document: https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token

to create a service principal and use it to access Databricks. I already have a databricks workspace configured and have used it to create a cluster. Then I've followed the process mentioned in the document, created a service principal and obtained the two tokens: AD Access token and management access token. However, I am unable to use the API.

The final call cURL command after configuration:

curl -X GET \
-H 'Authorization: Bearer <access-token>' \
-H 'X-Databricks-Azure-SP-Management-Token: <management-access-token>' \
-H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Databricks/workspaces/<workspace-name>' \
https://<databricks-instance>/api/2.0/clusters/list

With the tokens and the other info substitutued provides the following result:

<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 403 User not authorized.</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /api/2.0/clusters/list. Reason:
<pre>    User not authorized.</pre></p>
</body>
</html>

I have tried a couple of things to resolve this, namely changing the original access token request to use the interactive flow using the authorization_code grant type and so on, but that just gives me an Invalid access token error.

Is there something wrong with the above configuration? Am I missing some permissions?

Update: Came to the Access control (IAM) page and the app does not seem to be included there.

Upvotes: 6

Answers (4)

RSW

Reputation: 1396

Issue:

In my case, I was using Azure Service principal to deploy ADB using databrickslabs/databricks provider. But it was throwing error while terraform plan.

Error: Response from server (403 Forbidden) User not authorized.: invalid character 'U' looking for beginning of value. 
Using azure-client-secret auth: host=https://adb-xxx.6.azuredatabricks.net, azure_workspace_resource_id=/subscriptions/xxx/resourceGroups/myrg/providers/Microsoft.Databricks/workspaces/adb-wksp01, azure_client_secret=***REDACTED***, azure_client_id=yyy, azure_tenant_id=zzz

Cause:

The service principal used for terraform deployment was deactivated. When a service principal is deactivated:

The service principal cannot authenticate to the workspaces from any method.
The service principal’s status shows as Inactive in the workspace admin setting page.

Solution:

Enabling Service principal again in workspace admin setting page fixed the issue for me.

As a workspace admin, log in to the Azure Databricks workspace.
Click your username in the top bar of the Azure Databricks workspace and select Settings.
Click on the Identity and access tab.
Next to Service principals, click Manage.
Select the service principal you want to activate. Under Status, check Active.

Reference:

Deactivate a service principal in your Azure Databricks workspace

Upvotes: 0

imtiyaz shaikh

Reputation: 11

I also faced the same error and issue persisted in SPN configuration in databricks. Basically, wrong client id was mention while granting access to SPN in databricks.

How to verify?

In databricks: Go to settings > Identity and Access > Service principals > Then verify 'Application Id' is matching in Azure portal

In azure portal: Search for SPN and go it Application (not SPN). Verify 'Application (client) ID' is matching with databricks from previous step.

Once I sync up this, I was able to use SPN to make successful API call.

Upvotes: 1

Shivam Sharma

Reputation: 17

In case you wish to access the Databricks endpoints with just the access token, as is the case with using DBX in CI/CD workflows to trigger the Databricks pipelines, you would need to add the service principal as a user in the Databricks workspace. So then only the access token would be needed.

Refer to the documentation here- Add Service Principal as Databricks user

Upvotes: 0

Carl Zhao

Reputation: 9549

You must grant a role to the service principal.

This is the result of my test that I did not grant a role to the service principal. The error is the same as yours:

Next, grant roles to the service principal according to the following process:

Azure Portal>Azure Databricks>Azure Databricks Service>Access control (IAM)>Add a role assignment>Select the role you want to grant and find your service principal>save

Finally, use the service principal to get the token.(Don’t forget to grant permissions to service principals and grant administrator consent)

Get an Azure Active Directory access token: Get the Azure Management Resource endpoint token: Use the management endpoint access token to access the Databricks REST API:

Upvotes: 10