CODEWITHSUNDEEP
amazon-web-servicesamazon-cloudwatchaws-glue
Billa
Billa

Reputation: 69

AWS Glue jobs status Dashboard

In our project total 10 Glue jobs are running daily. I would like to build a dashboard to show last 7 days jobs status it means either succeeded or failure. Tried to achieve it in CloudWatch with metrics, but not able do it. Please give an idea to build this dashboard.

Upvotes: 2

Views: 2097

Answers (1)

ettefette
ettefette

Reputation: 51

Probably a little late for the original questioner, but maybe helpful for others.

We had a similar task in our project. We have many jobs and need to monitor success and failure. In our experience, the built-in metrics aren't really reliable, nor do they really answer the question of whether a job was successful or not. But we found a good way for us by generating custom metrics in a generic way for all jobs. This also works for existing jobs afterwards without having to change the code. I wrote an article about it: https://medium.com/@ettefette/metrics-for-aws-glue-jobs-as-you-know-them-from-lambda-functions-e5e1873c615c

We have set cloudwatch alerts based on these metrics and we use the metrics in our grafana dashboard to monitor the glue jobs.

Upvotes: 1

Related Questions