CODEWITHSUNDEEP
google-kubernetes-enginegithub-actionskustomize
Heikkisorsa
Heikkisorsa

Reputation: 905

Use different environments in Github Actions in combination with GKE and Kustomize

I started with the Github Action template from Google for Deployment to GKE. Everything worked as it should and now i want to extend the functionality for different environments (i.e. a push on the master branch goes to prod and one on dev should deploy to stage).

In the template this should be in the very last section of the job:

    # Deploy the Docker image to the GKE cluster
- name: Deploy
  run: |-
    ./kustomize edit set image gcr.io/PROJECT_ID/IMAGE:TAG=gcr.io/$PROJECT_ID/$IMAGE:$GITHUB_SHA
    ./kustomize build . | kubectl apply -f -
    kubectl rollout status deployment/$DEPLOYMENT_NAME
    kubectl get services -o wide

I changed the kustomize build line to:

kubectl kustomize kubernetes/overlays/stage | kubectl apply -f -

This because my folder structure for the kustomize files looks like this:

+---kubernetes
|   +---base
|   |       deployment.yml
|   |       kustomization.yml
|   |       service.yml
|   |
|   \---overlays
|       +---prod
|       |       deployment.yml
|       |       kustomization.yml
|       |       service.yml
|       |
|       \---stage
|               deployment.yml
|               kustomization.yml
|               service.yml

And for the start all the deplyoment.yml and service.yml are still the same as before I added the environment structure, only the kustomization.yml in the overlays changed to:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

bases:
- ../../base

patchesStrategicMerge:
  - deployment.yml
  - service.yml

When I now run the Github Action workflow the deployment failes with the error:

Error: Missing kustomization file 'kustomization.yaml'.

When using local kubectl command it produces the consolidated output of the wanted environment:

kubectl kustomize kubernetes/overlays/stage

Upvotes: 1

Views: 718

Answers (2)

instance
instance

Reputation: 98

We ended up not using kustomize directly for this altogether since kustomize edit set does not take a path as argument or option. Instead we echoed the image details into the base/kustomization.yaml manually.

echo "images:\n  - name: eu.gcr.io/project-id/image\n    newName: eu.gcr.io/$PROJECT_ID/$IMAGE\n    newTag: \"$GITHUB_SHA\""
>> ./deployment/base/kustomization.yaml

we only set image info (eu.gcr.io/project-id/image) in our base/deployment.yaml and omit setting images in the overlays

Then it's possible to run e.g. kubectl kustomize ./kubernetes/overlays/prod or apply it to a cluster directly by using kubectl apply -k ./kubernetes/overlays/prod

Upvotes: 1

Heikkisorsa
Heikkisorsa

Reputation: 905

I needed in the root directory another file name kustomization.yaml containing all the resources:

resources:
- ./kubernetes/overlays/stage
- ./kubernetes/overlays/prod

This way it works.

Upvotes: 2

Related Questions