How to enable Gitlab CI/CD for Private GKE cluster?

Question

I would like to setup the AutoDevops functionality of Gitlab CI/CD, and for that I am trying to setup the existing kubernetes cluster as my environment.

However, the Gitlab requires Kubernetes Master API URL that uses to access the Kubernetes API. Kubernetes exposes several APIs, we want the "base" URL that is common to all of them, e.g., https://kubernetes.example.com rather than https://kubernetes.example.com/api/v1. we will get the API URL by running this command:

kubectl cluster-info | grep 'Kubernetes master' | awk '/http/ {print $NF}

which returns a https://

in my case, I have a private Ip which is https://172.10.1.x

There aren't any documentations to help setup the gitlab CI for a Private GKE cluster.

How can I set the gitlab to access my kubernetes master with the help of a running VM instance or a pod's service IP ? or if there are any solutions/workarounds suggestions to achieve this, please help.

Add Existing GKE cluster as Environment

Answer 1

There is now (Sept. 2020) an alternative, but it was not free (GitLab.com Premium/Ultimate only), is free in part in 14.5+ (Nov. 2021). Then fully with 15.3 (Aug. 2022)

See GitLab 13.4

Introducing the GitLab Kubernetes Agent

GitLab’s Kubernetes integration has long enabled deployment to Kubernetes clusters without manual setup. Many users have enjoyed the ease-of-use, while others have run into some challenges.

The current integration requires your cluster to be open to the Internet for GitLab to access it. For many organizations, this isn’t possible, because they must lock down their cluster access for security, compliance, or regulatory purposes. To work around these restrictions, users needed to create custom tooling on top of GitLab, or they couldn’t use the feature.

Today, we’re announcing the GitLab Kubernetes Agent: a new way to deploy to Kubernetes clusters. The Agent runs inside of your cluster, so you don’t need to open it to the internet. The Agent orchestrates deployments by pulling new changes from GitLab, rather than GitLab pushing updates to the cluster. No matter what method of GitOps you use, GitLab has you covered.

Note this is the first release of the Agent. Currently, the GitLab Kubernetes Agent has a configuration-driven setup, and enables deployment management by code. Some existing Kubernetes integration features, such as Deploy Boards and GitLab Managed Apps, are not yet supported. Our vision is to eventually implement these capabilities, and provide new security- and compliance-focused integrations with the Agent.

https://about.gitlab.com/images/13_4/gitops-header.png -- Introducing the GitLab Kubernetes Agent

See Documentation and Issue.

---

See also GitLab 13.5 (October 2020)

Install the GitLab Kubernetes Agent with Omnibus GitLab

Last month we introduced the GitLab Kubernetes Agent for self-managed GitLab instances installed with Helm.

This release adds support for the official Linux package.

In this new Kubernetes integration, the Agent orchestrates deployments by pulling new changes from GitLab, rather than GitLab pushing updates to your cluster.

You can learn more about how the Kubernetes Agent works now and check out our vision to see what’s in store.

See Documentation and Issue.

---

This is confirmed with GitLab 13.11 (April 2021):

GitLab Kubernetes Agent available on GitLab.com

The GitLab Kubernetes Agent is finally available on GitLab.com. By using the Agent, you can benefit from fast, pull-based deployments to your cluster, while GitLab.com manages the necessary server-side components of the Agent.

The GitLab Kubernetes Agent is the core building block of GitLab’s Kubernetes integrations.
The Agent-based integration today supports pull-based deployments and Network Security policy integration and alerts, and will soon receive support for push-based deployments too.

Unlike the legacy, certificate-based Kubernetes integration, the GitLab Kubernetes Agent does not require opening up your cluster towards GitLab and allows fine-tuned RBAC controls around GitLab’s capabilities within your clusters.

See Documentation and issue.

---

See GitLab 14.5 (November 2021)

GitLab Kubernetes Agent available in GitLab Free

Connecting a Kubernetes cluster with the GitLab Kubernetes Agent simplifies the setup for cluster applications and enables secure GitOps deployments to the cluster.

Initially, the GitLab Kubernetes Agent was available only for Premium users.

In our commitment to the open source ethos, we moved the core features of the GitLab Kubernetes Agent and the CI/CD Tunnel to GitLab Free.
We expect that the open-sourced features are compelling to many users without dedicated infrastructure teams and strong requirements around cluster management.
Advanced features remain available as part of the GitLab Premium offering.

See Documentation and Epic.

---

See GitLab 14.8 (February 2022)

The agent server for Kubernetes is enabled by default

The first step for using the agent for Kubernetes in self-managed instances is to enable the agent server, a backend service for the agent for Kubernetes. In GitLab 14.7 and earlier, we required a GitLab administrator to enable the agent server manually. As the feature matured in the past months, we are making the agent server enabled by default to simplify setup for GitLab administrators. Besides being enabled by default, the agent server accepts various configuration options to customize it according to your needs.

See Documentation and Issue.

---

With GitLab 15.3 (August 2022):

GitOps features are now free

When you use GitOps to update a Kubernetes cluster, also called a pull-based deployment, you get an improved security model, better scalability and stability.

The GitLab agent for Kubernetes has supported GitOps workflows from its initial release, but until now, the functionality was available only if you had a GitLab Premium or Ultimate subscription. Now if you have a Free subscription, you also get pull-based deployment support. The features available in GitLab Free should serve small, high-trust teams or be suitable to test the agent before upgrading to a higher tier.

In the future, we plan to add built-in multi-tenant support for Premium subscriptions. This feature would be similar to the impersonation feature already available for the CI/CD workflow.

See Documentation and Issue.

---

See GitLab 15.4 (September 2022)

Deploy Helm charts with the agent for Kubernetes

You can now use the agent for Kubernetes to deploy Helm charts to your Kubernetes cluster.

Until now, the agent for Kubernetes only supported vanilla Kubernetes manifest files in its GitOps workflow.
To benefit from the GitOps workflow, Helm users had to use a CI/CD job to render and commit resources.

The current release ships with Alpha support for Helm.
Because Helm is a mature product, we consider the solution performant. However, known issues exist and the API might change without prior notice. We welcome your feedback in the related epic, where we discuss future improvements and next steps.

See Documentation and Issue.