CODEWITHSUNDEEP
azure-functionsazure-application-insights
Kenny_I
Kenny_I

Reputation: 2513

How to optimize Application Insight costs for Azure Functions?

I have Azure Function(Python) and Azure Application Insight. Application Insight costs are huge, because it get 200GB data by month. I'm trying to track where is the issue.

I wonder Azure Function logs everything and is it possible to optimize. We would not need "info".

How to optimize costs in Application Insight for Azure Functions?

Application Insight:

logger = logging.getLogger(__name__)
    logger.info('Python HTTP trigger function received a request.')

    try:
        instrumentationKey = os.environ['APPINSIGHTS_INSTRUMENTATIONKEY']
        logger.addHandler(AzureLogHandler(
            connection_string=f'InstrumentationKey={instrumentationKey}')
        )
    except Exception as e:
        exception_text = f"{e}"
        logging.error(f"Could not add Application Insights logging: {exception_text}")

Logging usage:

 logging.error(f"EXCEPTION: {exception_text}")
 logging.info(f" Calling Activity Function")

Upvotes: 0

Views: 2203

Answers (2)

Anton Petrov
Anton Petrov

Reputation: 973

Reducing you observability because of costs is very unlucky. I was really disappointed by ApplInsights costs and started looking for an alternative immediately.

In my case with my requirements having a Loki with Grafana was enough, but my projects and the company are small.

P.S. I suspect AppInsights is very expensive because of its obsolete tech, I guess it uses a relational database which is very questionable desicion for logs and metrics.

Upvotes: 0

anon
anon

Reputation:

I wonder Azure Function logs everything and is it possible to optimize. We would not need "info".

1. Disable unneeded modules: Edit the ApplicationInsights.config to turn off collection modules which are not required.

  • Application Insights .NET SDK consists of a number of NuGet Packages that contains an API for sending telemetry and it also consists of telemetry modules and initializers for automatically tracking from your application and its context. You can enable or disable Telemetry Modules and initializers, and set parameters for some of them.

  • There's a node in the configuration file for each module. To disable a module, delete the node or comment it out.

2. Disable telemetry dynamically: To disable telemetry conditionally and dynamically anywhere in the code, set the DisableTelemetry flag on it using TelemetryConfiguration instance.

public void ConfigureServices(IServiceCollection services)
    {
        services.AddApplicationInsightsTelemetry();
    }

public void Configure(IApplicationBuilder app, IHostingEnvironment env, TelemetryConfiguration configuration)
{
    configuration.DisableTelemetry = true;
    ...
}

This code sample prevents the sending of telemetry to Application Insights but not the automatic collection modules from colleting telemetry and in order to remove auto collection modules also, please refer this Microsoft Documentation.

3. Customize Logs Collection:

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning"
    },
    "ApplicationInsights": {
      "LogLevel": {
        "Default": "Information"
      }
    }
  }
}

The following above configuration allows Application Insights to capture all Information logs and severe warning logs. To change this behavior, explicitly override the logging configuration for the provider ApplicationInsights as shown below:

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning"
    }
  }
}

There are few more techniques to manage the data volume used for telemetry data optimizing also like:

  • Sampling: Sampling is the primary tool used to tune the amount of data you send. Sampling in Application Insights
  • Daily Cap: The maximum cap in Application Insights is 1000 GB/day unless you request a higher maximum for a high-traffic application.
  • If you have workspace-based Application Insights resource, we recommend using the workspace's daily cap to limit ingestion and costs instead of the cap in Application Insights.
  • Pre-aggregate metrics: If you put calls to TrackMetric in your app, you can reduce traffic by using the overload that accepts your calculation of the average and standard deviation of a batch of measurements. Or, you can use a pre-aggregating package.

Please check these references for more information:

  1. Resolve if logs shows twice in Application Insights
  2. Optimizing logging costs of Azure Functions
  3. Configuring or removing the necessary Telemetry Initializers

Upvotes: 2

Related Questions