Reputation: 37
I am currently facing an issue with the Cloud Data Fusion (CDAP) post endpoint when attempting to gather metrics for a pipeline. The strange behavior is observed when hitting the endpoint from a Cloud Function compared to running it locally.
Here's the relevant code snippet:
headers = {
'Authorization': 'Bearer ' + creds.token
}
body = {
"query": {
"tags": {
"namespace": "default",
"app": {pipeline_name},
"workflow": "DataPipelineWorkflow",
"run": {latest_runid}
},
"metrics": [
"user.SQL.records.in",
"user.SQL.records.out",
"user.SQL.records.error",
"user.SQL.process.time.avg",
"user.BigQuery.records.in",
"user.BigQuery.records.out",
"user.BigQuery.process.time.avg"
],
"timeRange": {
"aggregate": True
}
}
}
metric_response = requests.post(f'{cdf_endpoint}/v3/metrics/query', json=body, headers=headers)
When running the code locally, I receive the following response:
{
"query": {
"startTime": 0,
"endTime": 1704996437,
"series": [
{
"metricName": "user.BigQuery.records.out",
"grouping": {},
"data": [
{
"time": 0,
"value": 100
}
]
},
{
"metricName": "user.BigQuery.process.time.avg",
"grouping": {},
"data": [
{
"time": 0,
"value": 39
}
]
},
{
"metricName": "user.SQL.records.out",
"grouping": {},
"data": [
{
"time": 0,
"value": 100
}
]
},
{
"metricName": "user.BigQuery.records.in",
"grouping": {},
"data": [
{
"time": 0,
"value": 100
}
]
}
],
"resolution": "2147483647s"
}
}
However, when the same code is executed from a Cloud Function, the response contains an empty series array:
{
"query": {
"startTime": 0,
"endTime": 1704996437,
"series": [],
"resolution": "2147483647s"
}
}
Can anyone help me with this?
Upvotes: 1
Views: 92