CODEWITHSUNDEEP
python-3.xazurepython-requestsazure-blob-storageazure-function-app
Meesam Ali
Meesam Ali

Reputation: 29

Pushing Azure Blob Storage Data directly into a POST request using Python

I am struggling with sending the data from Azure Blob to a platform that takes in a POST request to upload a file that should be in xlsx or csv format.

In order my code to work, the blob data need to be first written into a file using open() library and then reading the file back into the algorithm as io.BufferedReader <name=File_name.xlsx> which can then be used in the requests.post files parameter which is a json

PROBLEM: My problem is that i need to run this code in a production environment using azure function app. App doesnt allow me to simply use os and open() I have tried using io.BytesIO to get bytes data from the blob but the API doesnt recognize it as a csv or xlsx file

Is there a way to use the in memory data and use it for the POST request?

Here is my code:

# GETTING BLOB DATA AND UPLOADING IT USING POST REQUEST

from datetime import datetime, timezone, timedelta
import requests
import json
import os
from requests.auth import HTTPBasicAuth
from azure.storage.blob import BlobClient

# Get excel or csv file from azure blob storage (stored as blob)

# 1. Getting the blob data
connect_str = os.getenv("AZURE_STORAGE_CONNECTION_STRING") # Hard coding key when in production
container_name = 'CONTAINER_NAME'
blob_name = 'TEST.xlsx'
blob_client = BlobClient.from_connection_string(connect_str,container_name,blob_name)
data = blob_client.download_blob()

# Writing the Data into a file and then
# Reading back into the program to create
# io.BufferedReader <name='TEST.xlsx'> object for POST request

with open(blob_name,'wb') as file_content:
    file_content.write(data.readall())
    file_content.close()

f_data = open(blob_name,'rb')

_id = 'jkdfhiuweh93o0' # list_id for post request
file_name = "TEST"
url = "https://platform.com/api/uploadFile" # API endpoint
auth = {"X-Gatekeeper-SessionToken": "915760344f3a450e83e5fae98a78d87e"}
body = {"id": _id,'listname':file_name}

# Making a POST request and uploading the data
file_upload = requests.post(url, headers=auth, files=f_data, data=body)
print(file_upload.json())

I have been looking for a solution for weeks but no luck... Any comments and help is deeply appreciated.

Upvotes: 0

Views: 2119

Answers (1)

Meesam Ali
Meesam Ali

Reputation: 29

Finally, the way to use requests.post and embedding a file that is not obtained from online source is to use tempfile library.

Here is the solution

import tempfile, requests, json
from azure.storage.blob import BlobClient

token = {"X-Gatekeeper-SessionToken": "2444e9b6e3544cae6d8e9e6f4a5894"}
connect_str = '<Connection String - App Key 1>' # Obtained from azure storage account
container_name = '<Container Name>'
blob_name = "TEST.xlsx"
blob_client = BlobClient.from_connection_string(connect_str,container_name,blob_name)
data = blob_client.download_blob()

file_name = blob_name.split(".")[0]
file_format = blob_name.split(".")[-1]

# Temporarily Storing file in temp directory
temp_file = tempfile.NamedTemporaryFile('w+b', prefix=file_name, suffix="."+file_format)
temp_file.write(data.readall())
access_file = open(temp_file.name,'rb')

f_data = {'file':access_file,'Name':blob_name}

_id = 12345 #Each list (TEST.xlsx) has an id associated with it
if _id == 404:
    body = {"listname":file_name}
else:
    body = {"id": _id,'listname':file_name}

file_name = blob_name.split(".")[0]

url = "https://manage.fastfieldforms.com/api/lookuplist"


resp = requests.post(url, headers=token, files=f_data, data=body)
print(resp.json())
access_file.close()
temp_file.close()

Upvotes: 2

Related Questions