CODEWITHSUNDEEP
kubernetesargocdgitops
Zorem
Zorem

Reputation: 11

Managing dependencies between configuration and source repositories

To give you some background, we work in an architecture that separates the code into one repository and the helm chart (configuration) into another repository in accordance with the GitOps methodology.

Right now, as part of our CD process, there is a stage in gitlab ci that updates the helm chart value with the, newly built image, and applies all the changes made since the last deploy. This means that any changes we make to the helm chart won't be visible to k8s cluster until we deploy from pipeline that was created from the source repository. The good thing about the current scenario is that I have control over the changes that are made to our deploy environments, so I can avoid situations where a "version" of the config map is not aligned to the suitable image version thus prevent crashloop situations due to lack / unexpected change of configuration.

One issue that comes up when using Argo CD with autosync is that I can't plan a code change and a configuration update at the same time since if I change the values, Argo will modify the configmap and accordingly will also rollout the deployment (we have an annotation in the deployment whose value is the sha of the configmap so that the deployment will rollout with every change of the configmap through the helm) with the old version of the image.

**In summary, there are situations in which the source and the configuration are dependent on one another. **

One solution is to switch from autosync to manual sync. By doing so, we will be able to precisely control when the configmap change is applied and even trigger a sync when the CI process is complete after updating the image. In this arrangement, there won't be any inconsistency between the "version" of the configmap and image version.

The problem with this solution is that it significantly reduces the benefits of our use of Argo and pretty much leaves it in the same state where we currently work only with Argo's UI and some other nice features. It is best for us to continue on our current work path in this circumstance.

I would appreciate any suggestions for a fix or a general improvement. Perhaps we don't have a clear perspective on all of the gitops methodology.

As an aside, we are unable to use Argo rollouts at the moment.

Thank you!!

Upvotes: 1

Views: 548

Answers (2)

Ido Ziv
Ido Ziv

Reputation: 1

Tl;dr

Use auto sync, let argo install the manifest according to the template order, customize only if needed.

Follow this rule of order:

Permissions changes -> Configurations changes -> Image Change -> Ingress Changes

Deep dive

GitOps methodology with ArgoCD as the implementor is a great way of working, managing and mostly deploying to Kubernetes cluster. Using Auto Sync is an important feature for working in a GitOps methodology (from repo to server), as ArgoCD said here:

A benefit of automatic sync is that CI/CD pipelines no longer need direct access to the Argo CD API server to perform the deployment.

Let's break things down a bit, first of all ArgoCD only uses helm for templating the chart and then install the manifest it generated (explained in docs )

When Helm templates the manifests it uses a certain order (starting from v3.1.0 it process manifests in file path order, then in order found in each file)

Let's take an example the order of helm install to understand what is the "recommended" way for installing manifests:

  • Namespace
  • ...
  • ConfigMap
  • ...
  • Deployment

We first want to change the configurations and then change compute resources (deployments, StatefulSets) so in your case if both changes are required let ArgoCD follow the best practices and sync resources by the natural order: Configuration -> Applications Deployment (image for example)

If by any reason you need to customize the order you can use Sync Waves: a cool feature from ArgoCD that helps you deploy changes in phases.

Disclaimer

from what I understand you have issues when deploying changes to both chart templates and application code at the same time, please elaborate more if I miss understood

Upvotes: 0

Elad Shmitanka
Elad Shmitanka

Reputation: 1

You have 2 options (AFAIK):

  1. If you manage the ArgoCD application itself as code, you can sync them together. how? for example, if you have these sources in the ArgoCD application:
      sources:
        - repoURL: helm repo URL
          chart: some chart
          targetRevision: 'chart version'
          helm:
            ignoreMissingValueFiles: true
            valueFiles:
              - '$values/default.yaml'
        - repoURL: https://github.com/helm-values.git
          targetRevision: master
          ref: values

Then you can use the exact SHA of the values git repo. Then the flow would be:

  • promote new helm chart
  • Update values
  • Update argo application with the new values SHA and new helm version
  1. Support backwards compatibility in the helm. When you make a change, "hide it" with a feature flag, and then when you update the values, enable the flag as well as update the application, later, you will probably need to clean up

Upvotes: 0

Related Questions