Metrics alert does not fire for AppInsights webTest

Question

Below is an abridged copy of my ARM template (the variables it's referencing are pretty self explanatory), I can see the availability test run successfully in the Application Insights Availability pane, however I never see the alert being fired in the Monitoring area of the portal.

 {
      "type": "Microsoft.Insights/webtests",
      "apiVersion": "2015-05-01",
      "name": "[variables('pingHealthCheckWebTestName')]",
      "location": "[resourceGroup().location]",
      "comments": "emea-nl-ams-az is West Europe, emea-gb-db3-azr is North Europe. Notably for historical reasons emea-ru-msa-edge is suprisingly UK South and emea-se-sto-edge is UK West. In the properties.Configuration.WebTest, the documentation states we should delete the Id of the WebTest element and Guid inside the Items->Request element. It's a little nasty having the XML embedded in the JSON (n.b. JSON can't have line breaks), the alernative would be be a build step, however this is overkill as the WebTest format is being deprecated https://devblogs.microsoft.com/devops/cloud-based-load-testing-service-eol/. Notably this will not impact the Application Insights Availability Checker for the forseable future: https://github.com/MicrosoftDocs/vsts-docs/issues/3793  ",
      "tags": {
        "[concat('hidden-link:', resourceId('microsoft.insights/components/', variables('appInsightsName')))]": "Resource",
        "[concat('hidden-link:', resourceId('Microsoft.Web/sites/', variables('webRestApplicationName')))]": "Resource"
      },
      "properties": {
        "SyntheticMonitorId": "[variables('pingHealthCheckWebTestName')]",
        "Name": "[variables('pingHealthCheckWebTestName')]",
        "Enabled": true,
        "Frequency": 300,
        "Timeout": 120,
        "Kind": "ping",
        "RetryEnabled": true,
        "Locations": [
          {
            "Id": "emea-nl-ams-azr"
          },
          {
            "Id": "emea-gb-db3-azr"
          },
          {
            "Id": "emea-ru-msa-edge"
          },
          {
            "Id": "emea-se-sto-edge"
          },
          {
            "Id": "emea-fr-pra-edge"
          }
        ],
        "Configuration": {
          "WebTest": "[concat('<WebTest Name=\"',  variables('pingHealthCheckWebTestName') ,'\"         Enabled=\"True\"         CssProjectStructure=\"\"         CssIteration=\"\"         Timeout=\"120\"         WorkItemIds=\"\"         xmlns=\"http://microsoft.com/schemas/VisualStudio/TeamTest/2010\"         Description=\"\"         CredentialUserName=\"\"         CredentialPassword=\"\"         PreAuthenticate=\"True\"         Proxy=\"default\"         StopOnError=\"False\"         RecordedResultFile=\"\"         ResultsLocale=\"\">        <Items>        <Request         Method=\"GET\"         Version=\"1.1\"         Url=\"', variables('pingHealthCheckUrl') ,'\"         ThinkTime=\"0\"         Timeout=\"120\"         ParseDependentRequests=\"False\"         FollowRedirects=\"True\"         RecordResult=\"True\"         Cache=\"False\"         ResponseTimeGoal=\"0\"         Encoding=\"utf-8\"         ExpectedHttpStatusCode=\"200\"         ExpectedResponseUrl=\"\"         ReportingName=\"\"         IgnoreHttpStatusCode=\"False\" />        </Items>        </WebTest>')]"
        }
      },
      "dependsOn": [ "[variables('appInsightsName')]" ]
    },
    {
      "name": "[variables('pingHealthCheckAlertName')]",
      "type": "Microsoft.Insights/metricAlerts",
      "apiVersion": "2018-03-01",
      "location": "global",
      "dependsOn": [
        "[resourceId('Microsoft.Insights/webtests', variables('pingHealthCheckWebTestName'))]"
      ],
      "tags": {
        "[concat('hidden-link:', resourceId('microsoft.insights/components', variables('appInsightsName')))]": "Resource",
        "[concat('hidden-link:', resourceId('Microsoft.Insights/webtests', variables('pingHealthCheckWebTestName')))]": "Resource"
      },
      "properties": {
        "description": "[concat('Alert for ', variables('webRestApplicationName'),  'web availability test')]",
        "severity": 1,
        "enabled": "[parameters('webAvailabilityTestAlertEnabled')]",
        "scopes": [
          "[resourceId('Microsoft.Insights/webtests',variables('pingHealthCheckWebTestName'))]",
          "[resourceId('Microsoft.Insights/components',variables('appInsightsName'))]"
        ],
        "evaluationFrequency": "PT1M",
        "windowSize": "PT15M",
        "templateType": 0,
        "criteria": {
          "odata.type": "Microsoft.Azure.Monitor.WebtestLocationAvailabilityCriteria",
          "webTestId": "[resourceId('Microsoft.Insights/webtests', variables('pingHealthCheckWebTestName'))]",
          "componentId": "[resourceId('Microsoft.Insights/components', variables('appInsightsName'))]",
          "failedLocationCount": 3
        },
        "actions": [
          {
            "actionGroupId": "[resourceId('Microsoft.Insights/actionGroups', variables('webAvailabilityActionGroupName'))]"
          }
        ]
      }
    },

Notably I have been careful to use the exported arm template, as the application insights documentation refers to the deprecated alertrules resource.

Also I have been careful to include the webTestId and componentId properties which are not documented nor* visible in the exported ARM template, but instead can be seen if you save the alert in the monitoring portal and intercept the http traffic.

*I've come across again a place where I did see webTestId documented in the azure monitor section of the docs (not the app insights specific docs).

However even after all of this tuning the alert is still not firing.

My current in progress attempt:

After intercepting the traffic when saving a Web Availability check in the portal UI, the call to the resource manager is surprisingly different when saving an alert in the monitor UI. I've only tried this once so far, but it does appear to be different:

Here's the snippet of the section observed intercepting it:

...
 "templateType": "CorrelatedEvents",
    "templateSpecificParameters": null,
    "criteriaSchemaId": "WebtestLocationAvailabilityCriteria",
    "criteria": {
      "webTestId": "[resourceId('Microsoft.Insights/webtests', variables('pingHealthCheckWebTestName'))]",
      "componentId": "[resourceId('Microsoft.Insights/components', variables('appInsightsName'))]",
      "failedLocationCount": 3
    },

Notice that it's missing the odata declaration instead has criteriaSchemaId and the templateType is CorrelatedEvents rather than 0.

So I'm seeing all sorts of variations when looking at different sources...

However this is resulting in an internal server error when deploying, which I'm currently investigating.

UPDATE:

I noticed that the Web Availability tests UI when talking to the resource manager API was using an interesting version number: "2017-09-01-preview" which isn't documented as a supported version for the metricsAlerts, attempting the same however from an ARM template resulted in the error "The response for resource had empty or invalid content." so I'm a little stumped.

Answer 1

@Alex As Samara mentioned, there is an undergoing issue with the exported arm templates. Please use the template available here.

I just created a deployment and it went fine. From your question it appears that an availability test already exists and you just want to create a metric alert. If you are looking to only create a metric alert, please remove the "Microsoft.Insights/webtests" resource type from your ARM template. On comparing your template with the above template, the metric alert should work fine.

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "appName": {
      "type": "string"
    },
    "pingURL": {
      "type": "string"
    },
    "pingText": {
      "type": "string",
      "defaultValue": ""
    },
    "actionGroupId": {
      "type": "string"
    }
  },
  "variables": {
    "pingTestName": "[concat('PingTest-', toLower(parameters('appName')))]",
    "pingAlertRuleName": "[concat('PingAlert-', toLower(parameters('appName')), '-', subscription().subscriptionId)]"
  },
  "resources": [
    {
      "name": "[variables('pingAlertRuleName')]",
      "type": "Microsoft.Insights/metricAlerts",
      "apiVersion": "2018-03-01",
      "location": "global",
      "tags": {
        "[concat('hidden-link:', resourceId('Microsoft.Insights/components', parameters('appName')))]": "Resource",
        "[concat('hidden-link:', resourceId('Microsoft.Insights/webtests', variables('pingTestName')))]": "Resource"
      },
      "properties": {
        "description": "Alert for web test",
        "severity": 1,
        "enabled": true,
        "scopes": [
          "[resourceId('Microsoft.Insights/webtests',variables('pingTestName'))]",
          "[resourceId('Microsoft.Insights/components',parameters('appName'))]"
        ],
        "evaluationFrequency": "PT1M",
        "windowSize": "PT5M",
        "templateType": 0,
        "criteria": {
          "odata.type": "Microsoft.Azure.Monitor.WebtestLocationAvailabilityCriteria",
          "webTestId": "[resourceId('Microsoft.Insights/webtests', variables('pingTestName'))]",
          "componentId": "[resourceId('Microsoft.Insights/components', parameters('appName'))]",
          "failedLocationCount": 2
        },
        "actions": [
          {
            "actionGroupId": "[parameters('actionGroupId')]"
          }
        ]
      }
    }
  ]
}

Also, please make sure you are using the right commands to deploy your template:

Connect-AzAccount

Select-AzSubscription -SubscriptionName <yourSubscriptionName>

New-AzResourceGroupDeployment -Name AvailabilityAlertDeployment -ResourceGroupName ResourceGroupofApplicationInsightsComponent `
  -TemplateFile availabilityalert.json -TemplateParameterFile availabilityalert.parameters.json

Application Insights is now a part of Azure Monitor and the documentation is getting revamped for Application Insights. Until then, please use the Azure Monitor documentation for latest Application Insights updates.

Answer 2

There is currently a known issue with exporting templates for metric alerts missing required parameters and that using these templates causes an internal server error. I'm also working with the doc author about getting the template you linked to updated to include all the required properties.

Could you send me an email at azcommunity(at)microsoft.com? Please include your subscription number and a link to this thread. Any additional debugging info you can provide from the error will also be helpful. That will allow me to take additional steps to help you resolve the issue.