CODEWITHSUNDEEP
jenkinskuberneteskubectl
Giorgos K.
Giorgos K.

Reputation: 179

Server-side Validation of Kubernetes yaml

I would like to do server-side validation of Kubernetes yaml files before applying them.

I know that in my Jenkins agent, I could use the following kubectl command for validating yaml files at the server-side but I am a bit concerned about access-control:

The Kubernetes documentation says the following:

Authorization for dry-run and non-dry-run requests is identical. Thus, to make a dry-run request, the user must be authorized to make the non-dry-run request.

I don't want any Jenkins agents to have super powers over my EKS cluster. A bad actor could use my Jenkins agent maliciously and apply any manifests they wanted. Right now for security/stability/management reasons, creating Kubernetes objects is done by a different system not Jenkins.

I checked a few other options but I can see drawbacks:

Do you have any ideas or are you aware of any validation tools that I could use in our CI system securely for the purpose of validating end-2-end Kubernetes yaml files?

Upvotes: 2

Views: 1086

Answers (3)

eyarz
eyarz

Reputation: 186

client-side validation (with kubectl --dry-run) has little value because it is missing some trivial schema misconfigurations.

you will get better results if you will perform the validation with kubeval (as you mentioned, you can skip CRDs) or you can use kubeconform that has CRD support.

Upvotes: 1

Giorgos K.
Giorgos K.

Reputation: 179

Unfortunately what I wanted to do initially is not feasible. Applying server-side validation on a manifest containing objects with dependencies (e.g a Namespace and a Deployment)is not possible. See this issue.

As a result, client side validation is the solution moving forward. Either it is with Kubeval or any other client-side mechanism. Kubeval won't validate CRDs but there is away of ignoring them so that the validation does not fail:

Currently kubeval relies on schemas generated from the Kubernetes API. This means it's not possible to validate resources using CRDs. Currently you need to pass a flag to ignore missing schemas, though this may change in a future major version.

Upvotes: 1

Max Lobur
Max Lobur

Reputation: 6040

I've been looking for this myself and did not find a sufficient tooling. However, there are few workarounds:

  • Deploy all objects to a temporary ci-job-id namespace in dev/stage clusters. They should be the same as a prod, but will not impose the security risks you mentioned. This gives an additional benefit - you can check if everything got created, all pods are running. It helps to catch issues like insufficient resource requests, missing images, misconfigured Service selectors, etc. Also it let's you add a smoke test on top.
  • Spin a small minikube with all the CRDs specifically for CI validations. This approach gives you less coverage, but it is much cheaper to maintain.

Upvotes: 2

Related Questions