CODEWITHSUNDEEP
dockerkubernetesgoogle-cloud-platformgoogle-kubernetes-engine
user6704961
user6704961

Reputation:

Mounting a kubeconfig file using go-client auth method to a container

I use Google Kubernetes Engine from my Mac & I used to mount my kubeconfig file in a container to use it from some program inside with a simple -v /Users/johndoe/.kube/config:/home/johndoe/.kube/config

Recently, some cloud providers, including GCP & AWS changed their kubeconfig auth methods to use the new client-go credential plugins.

See this blog post from GCP.

So now the kubeconfig looks like that :

- name: my-cluster
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      command: /opt/homebrew/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin/gke-gcloud-auth-plugin
      installHint: Install gke-gcloud-auth-plugin for use with kubectl by following
        https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
      provideClusterInfo: true

So of course, just mounting the kubeconfig file doesn't work anymore, because the container is looking for /opt/homebrew/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin/gke-gcloud-auth-plugin (which is a binary returning the token to stdout and that I can't mount either since the os archs differ).

So I get the following error when running any kubectl command.

Get "https://x.x.x.x/version": getting credentials: exec: fork/exec /opt/homebrew/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin/gke-gcloud-auth-plugin: no such file or directory

This seems like a pretty common use case, how can I work around that ?

Upvotes: 0

Views: 280

Answers (1)

user6704961
user6704961

Reputation:

I've found a way to "hack" this since I wasn't able to find a clean solution.

Simply call the auth plugin and send its output to a file :

/opt/homebrew/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin/gke-gcloud-auth-plugin > exec_credentials.json

Create a shell script to act as a stub (here authStub.sh)

#!/bin/sh

cat /tmp/exec_credentials.json

Then mount both the shell script and the credentials file with the docker run:

docker run \
    -v exec_credentials:/tmp/exec_credentials \
    -v authStub.sh:/opt/homebrew/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin/gke-gcloud-auth-plugin

And then it will work for one hour.

Upvotes: 0

Related Questions