CODEWITHSUNDEEP
argo-workflowsargoproj
Siddhant
Siddhant

Reputation: 581

How to pass different value of parameters for each Step in Argo Workflow DAG

I have sample Argo DAG as mentioned below. As seen below, I have hardcoded the parameter values for each of the tasks. Each task uses a different parameter value. I know that Argo provides a way to submit a parameter file either in JSON or YAML format to pass the parameter values dynamically. However, I was wondering if there is a way to pass a different value for each of the tasks in the DAG below via the parameter file.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-diamond-custom-
spec:
  entrypoint: diamond
  templates:
  - name: echo
    retryStrategy:
      limit: 3
    inputs:
      parameters:
      - name: message
      - name: task
    container:
      image: index_ruby:latest
      imagePullPolicy: Never
      command: [ruby, "index.rb", "-s" , "{{inputs.parameters.task}}" , "-r", "{{inputs.parameters.message}}"]
  - name: diamond
    dag:
      tasks:
      - name: A
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "valueA"}
            - name: task
              value: A
      - name: B
        dependencies: [A]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "valueB"}
            - name: task
              value: B
      - name: C
        dependencies: [A]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "valueC"}
            - name: task
              value: C
      - name: D
        dependencies: [B, C]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "valueD"}
            - name: task
              value: D

Upvotes: 5

Views: 7136

Answers (1)

crenshaw-dev
crenshaw-dev

Reputation: 8344

The Argo Workflow spec includes arguments where a number of global parameters can be defined.

You can define one parameter for each task and then access it via templating.

(I modified the echo container a bit because I was having trouble with the Ruby image.)

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-diamond-custom-
spec:
  entrypoint: diamond
  arguments:
    parameters:
    - name: valueA
    - name: valueB
    - name: valueC
    - name: valueD
  templates:
  - name: echo
    retryStrategy:
      limit: 3
    inputs:
      parameters:
      - name: message
      - name: task
    container:
      image: docker/whalesay:latest
      command: [echo]
      args: ["{{inputs.parameters.task}}", "{{inputs.parameters.message}}"]
  - name: diamond
    dag:
      tasks:
      - name: A
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "{{workflow.parameters.valueA}}"}
            - name: task
              value: A
      - name: B
        dependencies: [A]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "{{workflow.parameters.valueB}}"}
            - name: task
              value: B
      - name: C
        dependencies: [A]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "{{workflow.parameters.valueC}}"}
            - name: task
              value: C
      - name: D
        dependencies: [B, C]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "{{workflow.parameters.valueD}}"}
            - name: task
              value: D

As you mentioned, you can create a JSON file of parameters...

{
  "valueA": 1,
  "valueB": 2,
  "valueC": 3,
  "valueD": 4
}

...and then pass the parameters from the file via argo submit.

argo submit workflow.yaml -f params.json --watch

Upvotes: 5

Related Questions