CODEWITHSUNDEEP
pythonazureazure-functionspickle
K S
K S

Reputation: 45

Load pickle file in Azure function from Azure Blob Storage

I try to build service via Azure functions that performs a matrix multiplication using a vector given by the http request and a fix numpy matrix. The matrix is stored in Azure blob storage as a pickle file and I want to load it via an input binding. However I do not manage to load the pickle file. I am able to load plain text files. Right now my approach looks like this:

    def main(req: func.HttpRequest, blobIn: func.InputStream) -> func.HttpResponse:
       logging.info('Python HTTP trigger function processed a request.')
       matrix = pickle.loads(blobIn.read())
       vector = req.params.get('vector')
       result = matrix.dot(vector)
       return func.HttpResponse(json.dumps(result))

The error I get when running it that way is UnpicklingError: invalid load key, '\xef'. Another approach I tried after some googling was the following:

    def main(req: func.HttpRequest, blobIn: func.InputStream) -> func.HttpResponse:
       logging.info('Python HTTP trigger function processed a request.')
       blob_bytes = matrix.read()
       blob_to_read = BytesIO(blob_bytes)
       with blob_to_read as f: 
          A = pickle.load(f)
       vector = req.params.get('vector')
       result = matrix.dot(vector)
       return func.HttpResponse(json.dumps(result))

But it yields the same error. I also tried to save the matrix in a text file, get the string and build the matrix based on the string, but I encountered other issues. So how can I load a pickle file in my Azure function? Is it even the correct approach to use input bindings to load such files or is there a better way? Many thanks for your help!

Upvotes: 2

Views: 3305

Answers (1)

suziki
suziki

Reputation: 14098

Thanks for evilSnobu's contribution.

So when face this problem, that means the pickle file you get in your code is corrupt.

The solution is add "dataType": "binary" to the input binding in function.json.

Like this:

{
  "name": "inputBlob",
  "type": "blob",
  "dataType": "binary",
  "direction": "in",
  "path": "xxx/xxx.xxx",
  "connection": "AzureWebJobsStorage"
}

Upvotes: 2

Related Questions