Reputation: 37
I want to implement a generic CI pattern using Argo Workflows. One Workflow is triggered by the git server using an Argo Events webhook. After validating the signature, the repo is cloned and a custom workflow from the git repo is started (with a fixed name, e.g. .argo-ci.yaml
). The Workflow of Workflows pattern is recommended to be implemented using the template.resource field. When using the manifest
key, I can use input variables as shown in the example. However, variables are not evaluated using manifestFrom
, which I would need to start the Workflow from git.
For a CI use case, the child workflow either needs to be passed the git repo artifact or at least the git revision as a variable so that the Workflow can clone and checkout the correct sources for which the CI was triggered. Both does not seem to be possible as the manifest from template.resource.manifestFrom
is applied as-is without templating options.
Upvotes: 1
Views: 399
Reputation: 11
I was trying to figure out a similar approach. I ended up with this the following workflow of workflows. I have a webhook that triggers an argo event creating a workflow from this template passing in the webhook header and body. It basically is three steps:
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
name: workflows-of-workflows-template
spec:
entrypoint: main
volumeClaimTemplates:
- metadata:
name: "{{workflow.name}}"
spec:
storageClassName: 'temp-nfs-csi'
accessModes: [ 'ReadWriteMany' ]
resources:
requests:
storage: 1Gi
arguments:
parameters:
- name: request_header
- name: request_body
templates:
- name: main
steps:
- - name: clone-repo
templateRef:
name: git-cluster-template
template: git-clone
clusterScope: true
arguments:
parameters:
- name: url
value: "{{=jsonpath(workflow.parameters.request_body, '$.repository.clone_url')}}"
- name: ref
value: "{{=jsonpath(workflow.parameters.request_body, '$.repository.default_branch')}}"
- - name: get-workflow
template: get-workflow
- - name: run-nested-workflow
when: "{{=steps['get-workflow'].outputs.result != 'none'}}"
template: run-workflow
arguments:
parameters:
- name: workflow
value: '{{steps.get-workflow.outputs.result}}'
- name: get-workflow
script:
image: alpine
nodeSelector:
reyrey.env: dev
command: ['sh']
source: |
if [ -f /git-repo/.argo-ci.yaml ]; then
cat /git-repo/.argo-ci.yaml
else
echo none
fi
volumeMounts:
- name: '{{workflow.name}}'
mountPath: /git-repo
subPath: git-repo
- name: run-workflow
inputs:
parameters:
- name: workflow
resource:
action: create
manifest: |
{{inputs.parameters.workflow}}
I have a cluster workflow template for cloning repos that has credentials tied to it. Steps one and two could potentially be merged into one template, but I have it separate to reuse cloning within the cluster. Also, I am creating a temporary volume claim for holding the cloned repo that is named after the workflow name. This could be removed if just one template is passing outputs/results.
I am also receiving webhooks from a gitea instance, so the body/headers will be different depending on your triggering event, but the steps should be similar. I decided to just pass through the full header and body and use jsonpath to parse what I need in each individual workflow.
Because the way argo workflows runs I noticed that this will act as the base workflow so the workflow parameters passed to this are accessible within the created workflow. And, anything that needs to be created in the base workflow needs to be created here (volumes, secretes, hostAliases, serviceAccount, etc).
Upvotes: 0