CODEWITHSUNDEEP
jenkinsyamlkubernetes-helmgitops
Max N.
Max N.

Reputation: 1145

Change value in yaml file without changing the files structure

I'm thinking about setting up a GitOps CI/CD pipeline in my cluster with Jenkins and ArgoCD.

For a start I want to have a repository for my CI environment with some values files for the Helm charts of my applications.

One thing that I cannot really figure out is, how I can automatically edit the Helm values files without changing the whole structure of the files.

The Jenkins pipeline methods for reading and writing yaml files will fully recreate it and in the process re-format the whole file.

yq does not (seem to) re-order the keys, but it removes empty lines and comments, for example.

Only other thing that comes to my mind is using sed. But that feels kind-of wrong. And it might easily break. Like when I add a second key in another group with the same name. Or add a or remove keys.

Here's an example, to make it a bit more clear:

I have two repositories, one for my application, one for my CI, NI, ... configuration.

My application repo is not that important. Just some application. My config repository looks somewhat like this:

 .
...
├── ci
│   ├── app1
│   │   ├── Chart.yaml
│   │   ├── values.yaml
│   ├── app2
...
├── staging
│   ├── app1
│   │   ├── Chart.yaml
│   │   ├── values.yaml
│   ├── app2
...

The values.yaml file of app1 in the ci folder might look like this:

app1:
  image:
    tag: 1.33.7

  ingress:
    enabled: true
    hosts:
      - app1.my-internal-ci.company.com
  ...

The actual helm charts are somewhere else and here is only the configuration for the the applications on the environments.

ArgoCD watches this repo and takes care that the desired state (as specified by the state of the repo) and the actual state in the cluster match (see https://argoproj.github.io/cd/ for more on the tool).

As a last step my CI pipeline for app1, that builds a docker image from the code in my application repo, should update the values.yaml file in the ci folder in the configuration repo with the new version 1.33.8 of the application that was just build and push it as a new commit to the main branch.

Besides that, there are other values configured, like the ingress for example, that I update manually in the repo, if needed.

Since the files are updated automatically by the CI build pipeline and manually by a developer / DevOps engineer, I would like to be able to keep them easily readable by a human (order of the keys, newlines, comments, ...)

Is there any other way of achieving this?

Thanks in advance!

Upvotes: 0

Views: 3271

Answers (2)

Max N.
Max N.

Reputation: 1145

After some further digging I think the best option I have right now is yq. In newer versions (> 3.0.0; https://github.com/mikefarah/yq/issues/19) it should not remove any comments anymore. And with the newline-thing I can live, I think.

So if you have the files as mentioned above you could update your image tag with:

yq -i '.app1.image.tag = "1.33.8"' ci/app1/values.yaml

Worth mentioning: If you're using kustomize instead of Helm, there is a built-in for this: kustomize edit set image my-app=myregistry.io/my-platform/my-app:1.2.3.

Thanks for your ideas and suggestions!

Upvotes: 1

David Maze
David Maze

Reputation: 158977

You can straightforwardly use the helm install -f option here. There are two important details about it:

  1. The helm install -f values are used in addition to the values.yaml in the chart. You do not need to repeat the chart's values in your local settings.
  2. helm install -f takes any valid YAML file; but every valid JSON file is a valid YAML file, so if your CI tool can write out JSON, it can create a file that can be passed to helm install -f.

Indeed, the precise format of the YAML file doesn't really matter, so long as the right keys and sequences and mappings are there with the right structure.

In a Jenkins context, you can use the standard writeJSON function to produce a local values file:

// Only need to provide the specific values we need to set;
// the chart's values.yaml will be used for anything we don't
// directly provide here
helmValues = ['environment': 'production',
              'tag': GIT_COMMIT]
writeJSON file: 'values.deploy.yaml', json: helmValues

sh 'helm upgrade --install -n myproject myproject ./charts/myproject -f values.deploy.yaml'

Upvotes: 0

Related Questions