CODEWITHSUNDEEP
continuous-integrationargo-workflows
Pit Wegner
Pit Wegner

Reputation: 37

How do I pass values to an Argo Workflow of Workflows?

I want to implement a generic CI pattern using Argo Workflows. One Workflow is triggered by the git server using an Argo Events webhook. After validating the signature, the repo is cloned and a custom workflow from the git repo is started (with a fixed name, e.g. .argo-ci.yaml). The Workflow of Workflows pattern is recommended to be implemented using the template.resource field. When using the manifest key, I can use input variables as shown in the example. However, variables are not evaluated using manifestFrom, which I would need to start the Workflow from git.

For a CI use case, the child workflow either needs to be passed the git repo artifact or at least the git revision as a variable so that the Workflow can clone and checkout the correct sources for which the CI was triggered. Both does not seem to be possible as the manifest from template.resource.manifestFrom is applied as-is without templating options.

Upvotes: 1

Views: 413

Answers (1)

Robert Saul
Robert Saul

Reputation: 11

I was trying to figure out a similar approach. I ended up with this the following workflow of workflows. I have a webhook that triggers an argo event creating a workflow from this template passing in the webhook header and body. It basically is three steps:

  1. clone the given repo at the default branch from the request body
  2. read the argo-ci.yaml file from the repo
  3. create a resource from the argo-ci.yaml file if it exists
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: workflows-of-workflows-template
spec:
  entrypoint: main
  volumeClaimTemplates:
  - metadata:
      name: "{{workflow.name}}"
    spec:
      storageClassName: 'temp-nfs-csi'
      accessModes: [ 'ReadWriteMany' ]
      resources:
        requests:
          storage: 1Gi
  arguments:
    parameters:
      - name: request_header
      - name: request_body
  templates:
  - name: main
    steps:
    - - name: clone-repo
        templateRef:
          name: git-cluster-template
          template: git-clone
          clusterScope: true
        arguments:
          parameters:
          - name: url
            value: "{{=jsonpath(workflow.parameters.request_body, '$.repository.clone_url')}}"
          - name: ref
            value: "{{=jsonpath(workflow.parameters.request_body, '$.repository.default_branch')}}"
    - - name: get-workflow
        template: get-workflow
    - - name: run-nested-workflow
        when: "{{=steps['get-workflow'].outputs.result != 'none'}}"
        template: run-workflow
        arguments:
          parameters:
          - name: workflow
            value: '{{steps.get-workflow.outputs.result}}'
  - name: get-workflow
    script:
      image: alpine
      nodeSelector:
        reyrey.env: dev
      command: ['sh']
      source: |
        if [ -f /git-repo/.argo-ci.yaml ]; then
          cat /git-repo/.argo-ci.yaml
        else
          echo none
        fi
      volumeMounts:
      - name: '{{workflow.name}}'
        mountPath: /git-repo
        subPath: git-repo
  - name: run-workflow
    inputs:
      parameters:
      - name: workflow
    resource:
      action: create
      manifest: |
        {{inputs.parameters.workflow}}

I have a cluster workflow template for cloning repos that has credentials tied to it. Steps one and two could potentially be merged into one template, but I have it separate to reuse cloning within the cluster. Also, I am creating a temporary volume claim for holding the cloned repo that is named after the workflow name. This could be removed if just one template is passing outputs/results.

I am also receiving webhooks from a gitea instance, so the body/headers will be different depending on your triggering event, but the steps should be similar. I decided to just pass through the full header and body and use jsonpath to parse what I need in each individual workflow.

Because the way argo workflows runs I noticed that this will act as the base workflow so the workflow parameters passed to this are accessible within the created workflow. And, anything that needs to be created in the base workflow needs to be created here (volumes, secretes, hostAliases, serviceAccount, etc).

Upvotes: 0

Related Questions