CODEWITHSUNDEEP
.net-coreterraformazure-application-insightsazure-aks
komluk
komluk

Reputation: 599

Application Insights duplicated logs

I have a .net core API running under Azure Kubernetes service, with Application Insights provisioned with Terraform. When viewing Application Insights in the Azure portal, I see doubled requests/exceptions/traces. Each telemetry has the exact same timestamp, response time, and telemetry data. I decided to manually deploy another Application Insights instance to verify my logging implementation. It turned out that in the second instance my telemetry data are not duplicated. So my question is: what option in the definition of terraform made this happen? There are some options in the Applications Insights responsible for that (maybe sampling or something)?

Here's my terraform definition for the Application Insights

resource "azurerm_application_insights" "application_insights" {
  name                                  = data.instance_name
  resource_group_name                   = data.resource_group.rg.name
  location                              = data.resource_group.rg.location
  application_type                      = var.application_type
  daily_data_cap_in_gb                  = var.daily_data_cap_in_gb
  daily_data_cap_notifications_disabled = var.daily_data_cap_notifications_disabled
  retention_in_days                     = var.retention_in_days
  sampling_percentage                   = var.sampling_percentage
  disable_ip_masking                    = var.disable_ip_masking
  workspace_id                          = local.workspace_Id

  tags = local.tags
}

My AppInsights query and results for 2 different instances

requests
| summarize count() by timestamp
| order by count_ asc

enter image description here

enter image description here

Here's a resource configuration downloaded from the azure portal:

Provisioned with terraform:

"kind": "other",
"properties": {
    "Application_Type": "other",
    "SamplingPercentage": 50,
    "RetentionInDays": 30,
    "DisableIpMasking": true,
    "IngestionMode": "LogAnalytics",
    "publicNetworkAccessForIngestion": "Enabled",
    "publicNetworkAccessForQuery": "Enabled",
    "DisableLocalAuth": false,
    "ForceCustomerStorageForProfiler": false
}

Provisioned manually:

"kind": "web",
"properties": {
    "Application_Type": "web",
    "Flow_Type": "Redfield",
    "Request_Source": "IbizaAIExtension",
    "RetentionInDays": 90,
    "publicNetworkAccessForIngestion": "Enabled",
    "publicNetworkAccessForQuery": "Enabled"
}

Upvotes: 1

Views: 1161

Answers (1)

Kartik Bhiwapurkar
Kartik Bhiwapurkar

Reputation: 5159

• One of the reasons for the log duplication in application insights workspace deployed by you through terraform might be that the ‘application_type’ attribute isn’t case-sensitive which according to the latest terraform AzureRM provider v2.0 should be as if it isn’t then the resource will be destroyed and re-created in place. Also, including a lifecycle customization and mentioning the ‘application_type’ as an attribute to ignore in it will also help you when logs in the application insights workspace are being telemetered. Since it will help in creating a plan for update as these will not considered during the update to the Azure resource deployment but will surely be considered while creating the resources from scratch and new.

An example of the above is given below: -

  resource "azurerm_application_insights" "appinsights" {
  name                = "${local.prefix}-appname"
   resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
   application_type    = "web"

     tags = local.tags

    lifecycle {
     ignore_changes = [
      application_type
    ]
    }
  }

• You can also use the ‘lifecycle’ feature along with ‘create_before_destroy’ syntax in the terraform which will change a resource argument that cannot be updated in-place due to remote API limitations in which the terraform engine will instead destroy the existing object and then create a new replacement object with the new configured arguments.

For more information, please refer to the links below: -

https://gist.github.com/spettifer/55dc46bad2f002f1c7def76814ccbd42

https://www.terraform.io/language/meta-arguments/lifecycle

https://github.com/hashicorp/terraform-provider-azurerm/issues/5902

Upvotes: 0

Related Questions