CODEWITHSUNDEEP
amazon-web-servicesaws-cloudformationaws-glue
sanjay lalwani
sanjay lalwani

Reputation: 35

AWS CloudFormation template for Glue with multiple predicate conditions

I need to write code of AWS CloudFormation Template for AWS Glue, where I am creating single template for multiple Glue job. I added predicate condition, for second job onwards. Unfortunately though predicate accept multiple conditions, it accepts only one state for previous job, as in if previous job state succeeded or failed or stopped, still next job should run. But I need it to run in multiple state as in even if previous job has any of the succeeded/failed/stopped state next job should add.

If I give multiple conditions, it gives error saying duplicate job exist, as I am giving same job name in all condition

I wrote below code snippet in template

'''
Resources:
  WF:
    Type: AWS::Glue::Workflow
    Properties:
        Description: workflow for Parallel Jobs
        Name: Sequence_Job_Triggers
  
  SequenceTrigger:
    Type: AWS::Glue::Trigger
    Properties:
      Name: Job_Trigger_1
      Type: ON_DEMAND
      Description: Description
      WorkflowName: !Ref WF
      Actions:
        - JobName: job1
  SequenceTrigger:
    Type: AWS::Glue::Trigger
    Properties:
      Name: Job_Trigger_2
      Type: CONDITIONAL
      Description: Description
      WorkflowName: !Ref WF
      Predicate:
        Conditions:
          - LogicalOperator: EQUALS
            JobName: job1
            State: SUCCEEDED
      Actions:
        - JobName: job2 
'''

Upvotes: 0

Views: 1685

Answers (1)

Robert Kossendey
Robert Kossendey

Reputation: 7028

Can't you just create two triggers, at least this works for me:

 SucceededSequenceTrigger:
    Type: AWS::Glue::Trigger
    Properties:
      Name: Succeeded_Job_Trigger_2
      Type: CONDITIONAL
      Description: Description
      WorkflowName: !Ref WF
      Predicate:
        Conditions:
          - LogicalOperator: EQUALS
            JobName: job1
            State: SUCCEEDED
      Actions:
        - JobName: job2 

  FailedSequenceTrigger:
    Type: AWS::Glue::Trigger
    Properties:
      Name: Failed_Job_Trigger_2
      Type: CONDITIONAL
      Description: Description
      WorkflowName: !Ref WF
      Predicate:
        Conditions:
          - LogicalOperator: EQUALS
            JobName: job1
            State: FAILED
      Actions:
        - JobName: job2

Upvotes: 1

Related Questions