CODEWITHSUNDEEP
google-kubernetes-enginegcp-config-connector
Jty.tan
Jty.tan

Reputation: 868

GKE Config Connector asking for resource.type which does not seem to be mandatory

I've got this config connector resource set up for alerting policy


apiVersion: monitoring.cnrm.cloud.google.com/v1beta1
kind: MonitoringAlertPolicy
metadata:
  name: testing-bff-alert
  namespace: config-connector
spec:
  alertStrategy:
    autoClose: 604800s
  combiner: OR
  conditions:
    - conditionThreshold:
        aggregations:
          - alignmentPeriod: 120s
            crossSeriesReducer: REDUCE_MEAN
            perSeriesAligner: ALIGN_DELTA
        comparison: COMPARISON_GT
        duration: 0s
        filter: metric.type="logging.googleapis.com/user/bff-prod-response-times"
        thresholdValue: 50
        trigger:
          count: 1
      displayName: 'BFF Response Times [Mean]'
  displayName: 'BFF Response Times [Mean] above 0.8s'
  documentation:
    content: BFF Response times are too high
    mimeType: text/markdown
  enabled: true
  notificationChannels:
    - external: "projects/my-project/notificationChannels/17980363"
    - name: slack-notification
      namespace: config-connector

The problematic part when applying this, is that the server complains about the filter missing the resource.type filter.

Update call failed: error applying desired state: summary: Error updating AlertPolicy "my-project projects/my-project/alertPolicies/65948650919": googleapi: Error 400: Field alert_policy.conditions[0].condition_threshold.filter had an invalid value of "metric.type="logging.googleapis.com/user/bff-prod-response-times"": must specify a restriction on "resource.type" in the filter; see "https://cloud.google.com/monitoring/api/resources" for a list of available resource types.

Now, from the actual metrics UI, I know that specifying just the metric.type filter works, and that it does not need anything else to work. In the UI adding AND resource.type="metric" to the existing (and working) metric.type just results in an empty chart with no data. So clearly my choice of resource.type="metric" isn't right.

And when I extract the configuration in YAML form from the server using the gcloud CLI, that too only has the metric.type, and nothing else.

Does anybody have any ideas about why this might be the case and what I can do to fix it? Or what additional resource.type filter I could be using that will work? I've been looking at https://cloud.google.com/logging/docs/api/v2/resource-list#service-names and haven't have had much luck guessing what other resource.type I should be using

Thank you!

Upvotes: 0

Views: 173

Answers (1)

Jty.tan
Jty.tan

Reputation: 868

Ok, turns out that whilst the visual editor allows you to NOT specify the resource type, the configuration approach does require you to use it, and in this case, it's resource.type="k8s_container"

The way we figured this out was by looking through all of the resources, and realising that the log based metrics appear in both the "Unspecified" section, as well as the "Kubernetes Container" section...

enter image description here

Upvotes: 0

Related Questions