cloud_anny
cloud_anny

Reputation: 37

Issue with Cloud Data Fusion (CDAP) Post Endpoint - Empty Series Array in Cloud Function

I am currently facing an issue with the Cloud Data Fusion (CDAP) post endpoint when attempting to gather metrics for a pipeline. The strange behavior is observed when hitting the endpoint from a Cloud Function compared to running it locally.

Here's the relevant code snippet:

headers = {
    'Authorization': 'Bearer ' + creds.token
}

body = {
    "query": {
        "tags": {
            "namespace": "default",
            "app": {pipeline_name},
            "workflow": "DataPipelineWorkflow",
            "run": {latest_runid}
        },
        "metrics": [
            "user.SQL.records.in",
            "user.SQL.records.out",
            "user.SQL.records.error",
            "user.SQL.process.time.avg",
            "user.BigQuery.records.in",
            "user.BigQuery.records.out",
            "user.BigQuery.process.time.avg"
        ],
        "timeRange": {
            "aggregate": True
        }
    }
}

metric_response = requests.post(f'{cdf_endpoint}/v3/metrics/query', json=body, headers=headers)

When running the code locally, I receive the following response:

{
  "query": {
    "startTime": 0,
    "endTime": 1704996437,
    "series": [
      {
        "metricName": "user.BigQuery.records.out",
        "grouping": {},
        "data": [
          {
            "time": 0,
            "value": 100
          }
        ]
      },
      {
        "metricName": "user.BigQuery.process.time.avg",
        "grouping": {},
        "data": [
          {
            "time": 0,
            "value": 39
          }
        ]
      },
      {
        "metricName": "user.SQL.records.out",
        "grouping": {},
        "data": [
          {
            "time": 0,
            "value": 100
          }
        ]
      },
      {
        "metricName": "user.BigQuery.records.in",
        "grouping": {},
        "data": [
          {
            "time": 0,
            "value": 100
          }
        ]
      }
    ],
    "resolution": "2147483647s"
  }
}

However, when the same code is executed from a Cloud Function, the response contains an empty series array:

{
      "query": {
        "startTime": 0,
        "endTime": 1704996437,
        "series": [],
        "resolution": "2147483647s"
      }
    }

Can anyone help me with this?

Upvotes: 1

Views: 92

Answers (0)

Related Questions