CODEWITHSUNDEEP
pythonazureazure-functionsazure-storage-account
Alidemir
Alidemir

Reputation: 11

Python code on Azure Function app to convert csv files to excel on Azure storage account

I'm new to Python code and had to write it for a project at work. If any one help me to correct the below code i will be appreciated.

I have a Azure Storage account with several containers. I am using HttpTrigger Function to fire the Azure function app on Azure Data Factory. When the pipeline on ADF creates a CSV file , pipeline should be able to run the function app to get the filename as parametric and convert csv file to excel file on Azure Storage Account.

I run the code succesfully on my local but when i deploy it to Function app i get the error. I have been trying to run the code for weeks but without success.

--- __init__.py file

import os, uuid
from azure.storage.blob import  BlobServiceClient, BlobClient, ContainerClient
import pandas as pd
from io import StringIO,BytesIO
import azure.functions as func
import xlsxwriter
import logging
def main(req: func.HttpRequest) -> func.HttpResponse:
    logging.info('Python HTTP trigger function processed a request.')
    name= "1893_Item_20220206_err.csv" #'"' ,req.params.get('name'),'"'
    if name:    
        connect_str = os.getenv('AzureWebJobsStorage') 
        blob_service_client = BlobServiceClient.from_connection_string(connect_str)
        container_name = "test"
        blob_client = blob_service_client.get_blob_client(container=container_name, blob=name)
        excel_blob_client = blob_service_client.get_blob_client(container=container_name, blob="1893_Item_20220206_err.xlsx")
        blob = blob_client.download_blob()
        read_file = pd.read_csv (StringIO(blob.content_as_text()),sep="|")
        print(read_file)
        output = BytesIO()
        writer = pd.ExcelWriter(output, engine='xlsxwriter')
        read_file.to_excel (writer)
        writer.save()
        output.seek(0)
        workbook = output.read()
        excel_blob_client.upload_blob(workbook, overwrite=True)
    else:   
     return func.HttpResponse(
            "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",
             status_code=200
  )


-- function.json file
{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "authLevel": "anonymous",
      "type": "httpTrigger",
      "path":"test/{blobname}.csv",
      "direction": "in",
      "name": "req",
      "methods": [
        "get",
        "post"
      ]
    },
    {
      "type": "http",
      "direction": "out",
      "name": "$return"
    }
  ]
}

Upvotes: 1

Views: 1916

Answers (1)

rickvdbosch
rickvdbosch

Reputation: 15571

Looking at your code my best guess would be that the value for the name parameter you've hardcoded contains the .csv extension while the name parameter in the path will not. You have that as a static string in your definition: "path":"test/{blobname}.csv".

Either append the csv to the name parameter, or remove it from the path specification.

Next to that, the parameter name in the path is blobname and not name.

Alternative solution

This scenario could be accomplished using an Azure Blob storage trigger for Azure Functions. That way the added blob will already be available as an InputStream.

The Blob storage trigger starts a function when a new or updated blob is detected. The blob contents are provided as input to the function.

Here's an example (taken from the article linked to above

Function.json:

{
    "scriptFile": "__init__.py",
    "disabled": false,
    "bindings": [
        {
            "name": "myblob",
            "type": "blobTrigger",
            "direction": "in",
            "path": "samples-workitems/{name}",
            "connection":"MyStorageAccountAppSetting"
        }
    ]
}

and

import logging
import azure.functions as func


def main(myblob: func.InputStream):
    logging.info('Python Blob trigger function processed %s', myblob.name)

Upvotes: 1

Related Questions