Using single helm chart for deployment of multiple services

Question

I am new to helm and kubernetes.

My current requirement is to use setup multiple services using a common helm chart.

Here is the scenario.

1. I have a common docker image for all of the services

2. for each of the services there are different commands to run. In total there are more than 40 services.

   Example

pipenv run python serviceA.py 
pipenv run python serviceB.py 
pipenv run python serviceC.py  
and so on...

Current state of helm chart I have is

demo-helm
|- Chart.yaml
|- templates
   |- deployment.yaml
   |- _helpers.tpl
|- values
   |- values-serviceA.yaml
   |- values-serviceB.yaml
   |- values-serviceC.yaml
    and so on ...

Now, since I want to use the same helm chart and deploy multiple services. How should I do it?

I used following command helm install demo-helm . -f values/values-serviceA.yaml -f values-serviceB.yaml but it only does a deployment for values file provided at the end.

Here is my deployment.yaml file

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "helm.fullname" . }}
  labels:
    {{- include "helm.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      {{- include "helm.selectorLabels" . | nindent 6 }}
  template:
    metadata:
    {{- with .Values.podAnnotations }}
      annotations:
        {{- toYaml . | nindent 8 }}
    {{- end }}
      labels:
        {{- include "helm.selectorLabels" . | nindent 8 }}
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          command: {{- toYaml .Values.command |nindent 12}} 
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
          volumeMounts:
            - name: secrets
              mountPath: "/usr/src/app/config.ini"
              subPath: config.ini
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      volumes:
        - name: secrets
          secret:
            secretName: sample-application
            defaultMode: 0400

Update.

Since my requirement has been updated to add all the values for services in a single file I am able to do it by following.

deployment.yaml

{{- range $service, $val := .Values.services }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ $service }}
  labels:
    app: {{ .nameOverride }}
spec:
  replicas: {{ .replicaCount }}
  selector:
    matchLabels:
      app: {{ .nameOverride }}
  template:
    metadata:
      labels:
        app: {{ .nameOverride }}
    spec:
      imagePullSecrets:
        - name: aws-ecr
      containers:
        - name: {{ $service }}
          image: "image-latest-v3"
          imagePullPolicy: IfNotPresent
          command: {{- toYaml .command |nindent 12}}
          resources:
            {{- toYaml .resources | nindent 12 }}
          volumeMounts:
            - name: secrets
              mountPath: "/usr/src/app/config.ini"
              subPath: config.ini
      volumes:
        - name: secrets
          secret:
            secretName: {{ .secrets }}
            defaultMode: 0400
{{- end }}

and values.yaml

services:
  #Services for region1
  serviceA-region1:
    nameOverride: "serviceA-region1"
    fullnameOverride: "serviceA-region1"
    command: ["bash", "-c", "python serviceAregion1.py"]
    secrets: vader-search-region2
    resources: {}
    replicaCount: 5

  #Services for region2
  serviceA-region2:
    nameOverride: "serviceA-region2"
    fullnameOverride: "serviceA-region2"
    command: ["bash", "-c", "python serviceAregion2.py"]
    secrets: vader-search-region2
    resources: {}
    replicaCount: 5

Now I want to know will the following configuration work with the changes I am posting below for both values.yaml

services:
  region:
  #Services for region1
    serviceA-region1:
      nameOverride: "serviceA-region1"
      fullnameOverride: "serviceA-region1"
      command: ["bash", "-c", "python serviceAregion1.py"]
      secrets: vader-search-region2
      resources: {}
      replicaCount: 5
   
  region:2
  #Services for region2
    serviceA-region2:
      nameOverride: "serviceA-region2"
      fullnameOverride: "serviceA-region2"
      command: ["bash", "-c", "python serviceAregion2.py"]
      secrets: vader-search-region2
      resources: {}
      replicaCount: 5

and deployment.yaml

{{- range $region, $val := .Values.services.region }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ $region }}-{{ .nameOverride }}
  labels:
    app: {{ .nameOverride }}
spec:
  replicas: {{ .replicaCount }}
  selector:
    matchLabels:
      app: {{ .nameOverride }}
  template:
    metadata:
      labels:
        app: {{ .nameOverride }}
    spec:
      imagePullSecrets:
        - name: aws-ecr
      containers:
        - name: {{ $region }}-{{ .nameOverride }}
          image: "image-latest-v3"
          imagePullPolicy: IfNotPresent
          command: {{- toYaml .command |nindent 12}}
          resources:
            {{- toYaml .resources | nindent 12 }}
          volumeMounts:
            - name: secrets
              mountPath: "/usr/src/app/config.ini"
              subPath: config.ini
      volumes:
        - name: secrets
          secret:
            secretName: {{ .secrets }}
            defaultMode: 0400
{{- end }}

Answer 1

I can recommend you try a helmfile-based approach. I prefer a 3-file approach. What you'll need :

• helmfile-init.yaml: contains YAML instructions that you might need to use for creating and configuring namespaces etc.
• helmfile-backend.yaml: contains all the releases you need to deploy (service1, service2 ...)
• helmfile.yaml: paths to the above-mentioned (helmfile-init, helmfile-backend YAML files)
• a deployment spec file (app_name.json): a specification file that contains all the information regarding the release (release-name, namespace, helm chart version, application-version, etc.)

Helmfile has made my life a little bit breezy when deploying multiple applications. I will edit this answer with a couple of examples in a few minutes.

Meanwhile, you can refer to the official docs here or the Blue Books if you have Github access on your machine.

Answer 2

Is there any other approach like I run everything in a loop since the only difference in each of my values.yaml file is the command section. So, I can include command in the same file like this > command: > - ["bash", "-c", "python serviceA.py"] > - ["bash", "-c", "python serviceB.py"] > - ["bash", "-c", "python serviceC.py"] – whoami 20 hours ago

Yes, you can write a fairly simple bash script which will run everything in a loop:

for i in {A..Z}; do sed "s/{{COMMAND}}/[\"bash\", \"-c\", \"python service$i.py\"]/g" values/values-service-template.yaml | helm install demo-helm-$i . -f - ; done

Instead of command: ["bash", "-c", "python serviceAregion1.py"] in your values/values-service-template.yaml file just put command: {{COMMAND}} as it will be substituted with the exact command with every iteration of the loop.

As to {A..Z} put there whatever you need in your case. It might be {A..K} if you only have services named from A to K or {1..40} if instead of letters you prefer numeric values.

The following sed command will substitute {{COMMAND}} fragment in your original values/values-service-template.yaml with the actual command e.g. ["bash", "-c", "python serviceA.py"], ["bash", "-c", "python serviceB.py"] and so on.

sed "s/{{COMMAND}}/[\"bash\", \"-c\", \"python service$i.py\"]/g" values/values-service-template.yaml

Then it will be piped ( | symbol ) to:

helm install demo-helm-$i . -f -

where demo-helm-$i will be expanded e.g. to demo-helm-A but the key element here is - character which means: read from standard input instead of reading from file, which is normally expected after -f flag.

Answer 3

helm install demo-helm . -f values/values-serviceA.yaml -f values-serviceB.yaml

When you did like this, serviceB values will override serviceA values. You need to run the command separately with different release name as follow :

helm install demo-helm-A . -f values/values-serviceA.yaml
helm install demo-helm-B . -f values/values-serviceB.yaml