Metricbeat gcp module to capture bigquery metrics is not working as expected

Question

I have added bigquery metrics configuration in modules.d/gcp.yml file

But getting an error below

"2021-12-06T09:51:10.206Z ERROR [gcp.metrics] metrics/metricset.go:294 bigquery.googleapis.com/storage/table_count" metric descriptor is empty, this metric will not be collected

Below is the code which I have used

• module: gcp region: "us-central1" metricsets:
  • metrics project_id: "ProjectName" credentials_file_path: "/root/xyz.json" exclude_labels: false period: 1h metrics:
  • aligner: ALIGN_NONE service: bigquery service_metric_prefix: bigquery.googleapis.com/ metric_types:
    • "bigquery.googleapis.com/storage/table_count"

Let me know if there is any changes needed in code.

I have also tried extracting compute and billing metrics . We can see the column names populating in Kibana discovery filter area , but we do not have any data available

Please find the code given below

• module: gcp region: "us-central1" metricsets:
  • metrics project_id: "projectname" credentials_file_path: "/root/xyz.json" exclude_labels: false period: 1 metrics:
  • aligner: ALIGN_NONE service: compute metric_types:
    • "instance/cpu/reserved_cores"
    • "instance/cpu/usage_time"
    • "instance/cpu/utilization"
    • "instance/uptime"

Thanks

Answer 1

The code below is worked well for me.

- module: gcp
  metricsets:
    - metrics
  project_id: "YOUR_PROJECT_ID"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1800s # 1800s is recommended
  metrics:
    - aligner: ALIGN_NONE
      service: bigquery
      service_metric_prefix: bigquery.googleapis.com/ # This is not required as already "bigquery.googleapis.com/" is default value
      metric_types:
        - "storage/table_count" #only "storage/table_count" here

It seems that you should edit your metric_type attribute only.

I recommend the collection period of table_count metric to 1800s as GCP sampled this every 1800 seconds.

Refer here for the information about that.

This works well at my kibana too.