CODEWITHSUNDEEP
pythonapache-sparkaccess-tokenhttp-errorazure-databricks
user39602
user39602

Reputation: 359

Getting HTTP error 403 - invalid access token while trying to access cluster through Azure databricks

I'm trying to access Azure databricks spark cluster by a python script which takes token as an input generated via databricks user settings and calling a Get method to get the details of the cluster alongwith the cluster-id.

The below is the code snippet. As shown, I have created a cluster in southcentralus zone.

import requests
headers = {"Authorization":"Bearer dapiad************************"}
data=requests.get("https://southcentralus.azuredatabricks.net/api/2.0/clusters/get?cluster_id=**************",headers=headers).text
print data

Expected result should give the full detail of the cluster eg. {"cluster_id":"0128-******","spark_context_id":3850138716505089853,"cluster_name":"abcdxyz","spark_version":"5.1.x-scala2.11","spark_conf":{"spark.databricks.delta.preview.enabled":"true"},"node_type_id" and so on .....}

The above code is working when I execute the code on google colaboratory whereas the same is not working with my local IDE i.e. idle. It gives the error of HTTP 403 stating as below: 

<p>Problem accessing /api/2.0/clusters/get. Reason:
<pre>    Invalid access token.</pre></p>

Can anyone help me resolve the issue? I'm stuck on this part and not able to access the cluster through APIs.

Upvotes: 2

Views: 16300

Answers (1)

Venkataraman R
Venkataraman R

Reputation: 12959

It could be due to encoding issue when you pass the secret. Please look into this issue and how to resolve it. Even though the resolution they have given for AWS,it could be similar one for Azure as well. Your secret might be having "/", which you have to replace.

There is a known problem in the last update related to the '+' character in secret keys. In particular, we no longer support escaping '+' into '%2B', which some URL-encoding libraries do.

The current best-practice way of encoding your AWS secret key is simply

secretKey.replace("/","%2F")

sample python script is given below:

New_Secret_key = "MySecret/".replace("/","%2F")

https://forums.databricks.com/questions/6590/s3serviceexception-raised-when-accessing-via-mount.html

https://forums.databricks.com/questions/6621/responsecode403-responsemessageforbidden.html

Upvotes: 1

Related Questions