CODEWITHSUNDEEP
kubernetesgoogle-kubernetes-engine
gerasalus
gerasalus

Reputation: 7707

Access Kubernetes GKE cluster outside of GKE cluster with client-go?

I general for authenticating with kubernetes clusters from client-go I see that I have two options:

So it is easy to access clusterA from clusterA but not clusterB from clusterA.

What are my options here? It seems that I just cannot pass GOOGLE_APPLICATION_CREDENTIALS and hope that client-go will take care of itself.

So my thinking:

Is this the correct approach, or is there a simpler way? I see that tokens have an expiration date?

Update:

It seems I can also use CLOUDSDK_CONTAINER_USE_CLIENT_CERTIFICATE=True gcloud beta container clusters get-credentials clusterB --zone. Which would add certificates to kube conf which I could use. But AFAIK those certificates cannot be revoked

Upvotes: 5

Views: 3771

Answers (1)

ahmet alp balkan
ahmet alp balkan

Reputation: 45312

client-go needs to know about:

  1. cluster master’s IP address
  2. cluster’s CA certificate

(If you're using GKE, you can see these info in $HOME/.kube/config, populated by gcloud container clusters get-credentials command).

I recommend you to either:

  1. Have a kubeconfig file that contains these info for clusters A & B
  2. Use GKE API to retrieve these info for clusters A & B (example here) (You'll need a service account to do this, explained below.)

Once you can create a *rest.Config object in client-go, client-go will use the auth plugin that's specified in the kubeconfig file (or its in-memory equivalent you constructed). In gcp auth plugin, it knows how to retrieve a token.

Then, Create a Cloud IAM Service Account and give it "Container Developer" role. Download its key.

Now, you have two options:

Option 1: your program uses gcloud

gcloud auth activate-service-account --key-file=key.json
KUBECONFIG=a.yaml gcloud container clusters get-credentials clusterA
KUBECONFIG=b.yaml gcloud container clusters get-credentials clusterB

Then create 2 different *rest.Client objects, one created from a.yaml, another from b.yaml in your program.

Now your program will rely on gcloud binary to retrieve token every time your token expires (every 1 hour).

Option 2: use GOOGLE_APPLICATION_CREDENTIALS

  1. Don't install gcloud to your program’s environment.
  2. Set your key.json to GOOGLE_APPLICATION_CREDENTIALS environment variable for your program.
  3. Figure out a way to get cluster IP/CA (explained above) so you can construct two different *rest.Config objects for cluster A & B.
  4. Now your program will use the specified key file to get an access_token to Google API every time it expires (every 1h).

Hope this helps.

P.S. do not forget to import _ "k8s.io/client-go/plugin/pkg/client/auth/gcp" in your Go program. This loads the gcp auth plugin!

Upvotes: 12

Related Questions