CODEWITHSUNDEEP
pysparkazure-databricks
manish007
manish007

Reputation: 11

How to read data from Databricks DBFS using Rest API in csv or Excel format?

I am using Databricks Rest API to read datasets stored on DFBS. The output is coming in 64bit encoded format and in json format. I need the output in tabular format which is easy to read.

Output of Rest Api: { "bytes_read": 4601, "data": "U2VwYWxMZW5ndGgsU2VwYWxXaWR0aCxQZXRhbE" }

output needed:

enter image description here

Upvotes: 0

Views: 1188

Answers (1)

Priyank Sharma
Priyank Sharma

Reputation: 11

You can use below code to download csv file,

jsonbody = {"path": dbfspath}

TOKEN = "dapi....." # you can generate this token from users settings

response = requests.get('https://eastus.azuredatabricks.net/api/2.0/dbfs/read/', headers={'Authorization': 'Bearer %s' % self.TOKEN }, json= jsonbody )

filedata = json.loads(response.text)["data"]

with open(outputpath, "wb") as file:

  file.write(base64.b64decode(filedata))

Let me know if you need any other help

Upvotes: 1

Related Questions