Issue running and deploying Azure function since outage(MS_FUNCTION_LOGS 5,ScriptApplicationHostOptionsSetup,"","ScmRunFromPackage is empty

Question

I am using Azure and trying to deploy a function to Azure function app, which reads in a file from blob storage when triggered, processes it and then returns the output as a csv. I am receiving an error when trying to run the function locally, but also when trying to deploy it and have been stuck with this problem a few days so hoping someone can help please. Previously the function was running locally using the command 'func start', however since the Azure outage, I am now obtaining the following message:

❯ func startFound Python version 3.11.0 (python3).

Azure Functions Core ToolsCore Tools Version:       4.0.5907 

Commit hash: N/A <numbers> (64-bit)

Function Runtime Version: 4.834.3.22875

MS_FUNCTION_LOGS 5,,,,,ScriptApplicationHostOptionsSetup,"","ScmRunFromPackage is empty.",4.834.3.22875,2024-08-01T08:42:39.2650890Z,,"",,,,SAUPLOAD,,,,

The function opens and closes the trigger functionality before i am able to test it by uploading a file.

File layout:

project_root/

│
├── testing_data_transformation/
│   ├── \__init_\_.py
│   ├── main.py
│   ├── config.py
│   └── function.json
│
├── host.json
├── local.settings.json
└── requirements.txt

The content of the files:

host.json

{
"version": "2.0",
"logging": {
  "applicationInsights": {
    "samplingSettings": {
      "isEnabled": true,
      "excludedTypes": "Request"
    }
  },
  "logLevel": {
    "default": "Information"
  },
  "extensions": {
    "blobStorage": {
      "dynamicPort": false
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[2.*, 3.0.0)"
  }
}
}

config.py

# Import the necessary module
from dotenv import load_dotenv
import os
# Load environment variables from the .env file (if present)
load_dotenv()
# Access environment variables as if they came from the actual environment
CONNECTION_STRING = os.getenv('CONNECTION_STRING')
CONTAINER_NAME = os.getenv('CONTAINER_NAME')
ACS_CONNECTION_STRING = os.getenv('ACS_CONNECTION_STRING')
SENDER_EMAIL = os.getenv('SENDER_EMAIL')

function.json

{
    "scriptFile": "main.py",
    "entryPoint": "main",
    "bindings": [
      {
        "name": "myblob",
        "type": "blobTrigger",
        "direction": "in",
        "path": "manualupload/{name}",
        "connection": "AzureWebJobsStorage"
      }
    ]
  } 

local.settings.json

{
  "IsEncrypted": false,
  "Values": {
    "AZURE_STORAGE_CONNECTION_STRING": "<CONNECTION STRING>",
    "AZURE_STORAGE_CONTAINER_NAME": "saupload",
    "FUNCTIONS_WORKER_RUNTIME": "python",
    "FUNCTIONS_EXTENSION_VERSION": "~4",
    "AzureWebJobsStorage": "<SAME CONNECTION STRING AS SPECIFIED ABOVE>"
  },
  "ConnectionStrings": {}
}

main.py

from io import BytesIO
import logging
import pandas as pd
import azure.functions as func 
from data_transformations import process_sheets
from azure.storage.blob import BlobServiceClient
from azure.communication.email import EmailClient

from config import (
    CONNECTION_STRING,
    CONTAINER_NAME,
    ACS_CONNECTION_STRING,
    SENDER_EMAIL,
)


def main(myblob: func.InputStream):
blob_name = os.path.basename(myblob.name)
    base_name = os.path.splitext(blob_name)[0]
    logging.info("""Python Blob trigger function processed blob Name: %s""", base_name)

    # Connecting to azure storage & read in excel file
    excel_data = pd.ExcelFile(BytesIO(myblob.read()))

    df = process_sheets(excel_data) # data cleaning/transformations take place
    print(df)

    df.to_csv('df_processed.csv')


return None

I have tried running it locally and also deploying it to Azure Functions. The error message received locally when running with verbose is:

Found Python version 3.11.0 (python3).

Azure Functions Core Tools
Core Tools Version:       4.0.5907 Commit hash: N/A <number> (64-bit)
Function Runtime Version: 4.834.3.22875

[2024-08-01T14:09:51.306Z] Starting metrics publisher for container : saupload. Publishing endpoint is https://:31002/api/metrics
[2024-08-01T14:09:51.329Z] Initializing LinuxContainerActivityPublisher
[2024-08-01T14:09:51.329Z] Starting LinuxContainerActivityPublisher
MS_FUNCTION_LOGS 5,,,,,ScriptApplicationHostOptionsSetup,"","ScmRunFromPackage is empty.",4.834.3.22875,2024-08-01T14:09:51.3300120Z,,"",,,,SAUPLOAD,,,,
[2024-08-01T14:09:51.350Z] Building host: version spec: ~4, startup suppressed: 'False', configuration suppressed: 'False', startup operation id: '<numbers>'
[2024-08-01T14:09:51.382Z] Reading host configuration file '/Users/.../host.json'
[2024-08-01T14:09:51.383Z] Host configuration file read:
[2024-08-01T14:09:51.383Z] {
[2024-08-01T14:09:51.383Z]   "version": "2.0",
[2024-08-01T14:09:51.383Z]   "logging": {
[2024-08-01T14:09:51.383Z]     "applicationInsights": {
[2024-08-01T14:09:51.383Z]       "samplingSettings": {
[2024-08-01T14:09:51.383Z]         "isEnabled": true,
[2024-08-01T14:09:51.383Z]         "excludedTypes": "Request"
[2024-08-01T14:09:51.383Z]       }
[2024-08-01T14:09:51.383Z]     },
[2024-08-01T14:09:51.383Z]     "logLevel": {
[2024-08-01T14:09:51.383Z]       "default": "Information"
[2024-08-01T14:09:51.383Z]     },
[2024-08-01T14:09:51.383Z]     "extensions": {
[2024-08-01T14:09:51.383Z]       "blobStorage": {
[2024-08-01T14:09:51.383Z]         "dynamicPort": false
[2024-08-01T14:09:51.383Z]       }
[2024-08-01T14:09:51.383Z]     },
[2024-08-01T14:09:51.383Z]     "extensionBundle": {
[2024-08-01T14:09:51.383Z]       "id": "Microsoft.Azure.Functions.ExtensionBundle",
[2024-08-01T14:09:51.383Z]       "version": "[2.*, 3.0.0)"
[2024-08-01T14:09:51.383Z]     }
[2024-08-01T14:09:51.383Z]   }
[2024-08-01T14:09:51.383Z] }
[2024-08-01T14:09:51.405Z] FUNCTIONS_WORKER_RUNTIME set to python. Skipping WorkerConfig for language: java
[2024-08-01T14:09:51.405Z] FUNCTIONS_WORKER_RUNTIME set to python. Skipping WorkerConfig for language: powershell
[2024-08-01T14:09:51.405Z] FUNCTIONS_WORKER_RUNTIME set to python. Skipping WorkerConfig for language: node
[2024-08-01T14:09:51.407Z] Extension Bundle not loaded. Loading extensions from /Users/.../bin. BundleConfigured: False, PrecompiledFunctionApp: False, LegacyBundle: False, DotnetIsolatedApp: False, isLogicApp: False
[2024-08-01T14:09:51.407Z] Script Startup resetting load context with base path: '/Users/.../bin'.
[2024-08-01T14:09:51.408Z] Loading startup extension 'AzureStorageQueues'
[2024-08-01T14:09:51.434Z] Loaded extension 'AzureStorageQueues' (5.1.3.0)
[2024-08-01T14:09:51.440Z] Loading startup extension 'AzureStorageBlobs'
[2024-08-01T14:09:51.441Z] Loaded extension 'AzureStorageBlobs' (5.3.1.0)
[2024-08-01T14:09:51.443Z] Reading host configuration file '/Users/.../host.json'
[2024-08-01T14:09:51.443Z] Host configuration file read:
[2024-08-01T14:09:51.443Z] {
[2024-08-01T14:09:51.443Z]   "version": "2.0",
[2024-08-01T14:09:51.443Z]   "logging": {
[2024-08-01T14:09:51.443Z]     "applicationInsights": {
[2024-08-01T14:09:51.443Z]       "samplingSettings": {
[2024-08-01T14:09:51.443Z]         "isEnabled": true,
[2024-08-01T14:09:51.443Z]         "excludedTypes": "Request"
[2024-08-01T14:09:51.443Z]       }
[2024-08-01T14:09:51.443Z]     },
[2024-08-01T14:09:51.443Z]     "logLevel": {
[2024-08-01T14:09:51.443Z]       "default": "Information"
[2024-08-01T14:09:51.443Z]     },
[2024-08-01T14:09:51.443Z]     "extensions": {
[2024-08-01T14:09:51.443Z]       "blobStorage": {
[2024-08-01T14:09:51.443Z]         "dynamicPort": false
[2024-08-01T14:09:51.443Z]       }
[2024-08-01T14:09:51.443Z]     },
[2024-08-01T14:09:51.443Z]     "extensionBundle": {
[2024-08-01T14:09:51.443Z]       "id": "Microsoft.Azure.Functions.ExtensionBundle",
[2024-08-01T14:09:51.443Z]       "version": "[2.*, 3.0.0)"
[2024-08-01T14:09:51.443Z]     }
[2024-08-01T14:09:51.443Z]   }
[2024-08-01T14:09:51.443Z] }

When deploying I get the following message:

Connected!
2024-08-01T15:52:02Z   [Verbose]   Request successfully matched the route with name '' and template 'admin/warmup'
2024-08-01T15:52:08Z   [Information]   Host Status: {
  "id": "<xx>",
  "state": "Running",
  "version": "4.34.2.2",
  "versionDetails": "4.34.2+0a01f0a7e656797ed93280e588f88c29bfbfcf61",
  "platformVersion": "102.1.7.93",
  "instanceId": "<xx>",
  "computerName": "<xx>",
  "processUptime": 8815484,
  "functionAppContentEditingState": "Unknown"
}
2024-08-01T15:52:08Z   [Information]   Host Status: {
  "id": "<xx>",
  "state": "Running",
  "version": "4.34.2.2",
  "versionDetails": "4.34.2+0a01f0a7e656797ed93280e588f88c29bfbfcf61",
  "platformVersion": "102.1.7.93",
  "instanceId": "<xx>",
  "computerName": "<xx>",
  "processUptime": 8815529,
  "functionAppContentEditingState": "Unknown"
}

Answer 1

I just stumbled upon this question and had the same issue. I was using v2 already and couldnt fix it that way. However what I realized is that we both used an environment variable called container_name.

My local.settings.json looked like this:

{
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "python",
    "container_name": "some_value_here"
  }
}

What I did now was to simply rename my setting container_name to container_reference and everything was working again. I guess the runtime uses a container_name variable itself and it gets overriden.

Answer 2

Create a new Azure function app and deploy the function using the command func azure functionapp publish appName.

Use below code to read an Excel file from blob storage container processes it and then returns the output as a csv when the function is triggered.

import io
import logging
import os
from io import BytesIO
import tempfile

import azure.functions as func
import pandas as pd
from azure.storage.blob import BlobClient, BlobServiceClient, ContainerClient

def main(blob: func.InputStream, context: func.Context, outputblob: func.Out[bytes]):
    logging.info(f"Processing blob: {blob.name}, Size: {blob.length} bytes")

    # Read the Excel file from blob
    excel_data = blob.read()
    df = pd.read_excel(io.BytesIO(excel_data))

    # Process the data 
    df.columns = [col.upper() for col in df.columns]

    # Convert DataFrame to CSV
    csv_data = df.to_csv(index=False)

    # Write the CSV to a new blob
    blob_service_client = BlobServiceClient.from_connection_string("Connection_string")
    container_client = blob_service_client.get_container_client("Container")

    # Create a temporary file to upload
    with tempfile.NamedTemporaryFile(delete=False, suffix=".csv") as temp_file:
        temp_file.write(csv_data.encode('utf-8'))
        temp_file_path = temp_file.name

    # Upload the CSV to the output blob container
    with open(temp_file_path, "rb") as data:
        container_client.upload_blob(name=f"processed_{blob.name.replace('.xlsx', '.csv')}", data=data)

    logging.info(f"Processed file saved as 'processed_{blob.name.replace('.xlsx', '.csv')}'")

local.settings.json:

{
  "IsEncrypted": false,
  "Values": {
    "AZURE_STORAGE_CONTAINER_NAME": "Container_name",
    "FUNCTIONS_WORKER_RUNTIME": "python",
    "FUNCTIONS_EXTENSION_VERSION": "~4",
    "AzureWebJobsStorage": "<Container_Name>"
  },
  "ConnectionStrings": {}
}

function.json:

{
  "bindings": [
    {
      "name": "blob",
      "type": "blobTrigger",
      "direction": "in",
      "path": "input_blob/{name}",
      "connection": "AzureWebJobsStorage"
    },
    {
      "name": "outputblob",
      "type": "blob",
      "direction": "out",
      "path": "output_blob/{name}",
      "connection": "AzureWebJobsStorage"
    }
  ]
}

host.json:

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[4.*, 5.0.0)"
  }
}

Output:

• Able to read the excel and convert into csv file and save it in output blob.

(.venv) C:\Users\uname\pyfn1>func start
Found Python version 3.11.9 (py).

Azure Functions Core Tools
Core Tools Version:       4.0.5907 Commit hash: N/A +807e89766a92b14fd07b9f0bc2bea1d8777ab209 (64-bit)       
Function Runtime Version: 4.834.3.22875

[2024-08-05T10:37:55.957Z] Worker process started and initialized.

Functions:
        BlobTrigger: blobTrigger

For detailed output, run func with --verbose flag.
[2024-08-05T10:38:01.737Z] Host lock lease acquired by instance ID '000000000000000000000000F72731CC'.
[2024-08-05T10:38:28.978Z] Executing 'Functions.BlobTrigger' (Reason='New blob detected(LogsAndContainerScan): manualupload/test.xlsx', Id=ecc50a98-c2bf-437e-bef6-dc172f1568fc)
[2024-08-05T10:38:28.982Z] Trigger Details: MessageId: 7d5e83f7-9d39-4701-aa11-2575521ab83d, DequeueCount: 1, InsertedOn: 2024-08-05T10:38:28.000+00:00, BlobCreated: 2024-08-05T09:34:27.000+00:00, BlobLastModified: 2024-08-05T10:38:26.000+00:00
[2024-08-05T10:38:29.073Z] Processing blob: manualupload/test.xlsx, Size: None bytes
[2024-08-05T10:38:29.567Z] Request URL: 'https://kpstrg05.blob.core.windows.net/kpupload/processed_manualupload/test.csv'
Request method: 'PUT'
Request headers:
    'Content-Length': '88'
    'x-ms-blob-type': 'REDACTED'
    'If-None-Match': '*'
    'x-ms-version': 'REDACTED'
    'Content-Type': 'application/octet-stream'
    'Accept': 'application/xml'
    'User-Agent': 'azsdk-python-storage-blob/12.21.0 Python/3.11.9 (Windows-10-10.0.22631-SP0)'
    'x-ms-date': 'REDACTED'
    'x-ms-client-request-id': 'dab624b5-5316-11ef-ba88-98fa9b0d1c80'
    'Authorization': 'REDACTED'
A body is sent with the request
[2024-08-05T10:38:30.575Z] Response status: 201
Response headers:
    'Content-Length': '0'
    'Content-MD5': 'REDACTED'
    'Last-Modified': 'Mon, 05 Aug 2024 10:38:31 GMT'
    'ETag': '"0x8DCB53ABFD2D364"'
    'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'
    'x-ms-request-id': 'f36ca0b5-c01e-008c-1a23-e7a2b3000000'
    'x-ms-client-request-id': 'dab624b5-5316-11ef-ba88-98fa9b0d1c80'
    'x-ms-version': 'REDACTED'
    'x-ms-content-crc64': 'REDACTED'
    'x-ms-request-server-encrypted': 'REDACTED'
    'Date': 'Mon, 05 Aug 2024 10:38:30 GMT'
[2024-08-05T10:38:30.578Z] Processed file saved as 'processed_manualupload/test.csv'
[2024-08-05T10:38:30.632Z] Executed 'Functions.BlobTrigger' (Succeeded, Id=ecc50a98-c2bf-437e-bef6-dc172f1568fc, Duration=2352ms)

Portal:

• Deployed to Azure using VS Code:

Portal: