CODEWITHSUNDEEP
aws-glue
Lucent Xu
Lucent Xu

Reputation: 93

Where does aws glue job events store?

I have two jobs in aws Glue.The two jobs can both manually run successfully.And I defined a trigger which will trigger job2 to run when job1 run succeeded.But the trigger doesn't work. Does someone know where does job events store,or how to debug it?

Upvotes: 0

Views: 490

Answers (1)

Yuriy Bondaruk
Yuriy Bondaruk

Reputation: 4750

You have to create another on-demand trigger which starts job1.

This is how AWS Glue is designed now - dependent jobs are only started if the job which completes was started by a trigger (not run ad-hoc). To create a job dependency chain, start the first job in the chain with a schedule or on-demand trigger. The initial thinking was that we wanted to make jobs to be tested in isolation and once each job looks good, customers can create workflows and let them run.

AWS Glue sends events to CloudWatch so you can create a CloudWatch rule to invoke lambda, forward to SQS/SNS etc.

Upvotes: 1

Related Questions