CODEWITHSUNDEEP
multithreadingkubernetes
jtee
jtee

Reputation: 79

How does Kubernetes Admission Controller handle multiple simultaneous admission requests?

Assuming that there is only one admission controller Pod running, and the admission controller has a webhook that will be triggered by Pod deletion events.

Example Scenario

There are 2 Pods (Pod A and Pod B) within a namespace. 2 different users (Alice and Bob) perform Pod deletion at the exact same time, in which:

  1. Alice deletes the Pod A
  2. Bob deletes the Pod B

In this specific scenario, will the admission controller handle both the admission requests serially or in parallel? In other words, will the admission controller handle the admission request for Pod A before that of Pod B (or vice versa), or will it handle both the admission requests are the same time?

General Scenario

The admission requests are sent from the API Server to the admission controller. Generally speaking, will it be possible that multiple admission requests are sent to the admission controller at the exact same time?

And if so, will the admission controller handle them in parallel via some built-in parallelism mechanism, or will the admission controller queue them and process them serially?

Upvotes: 1

Views: 944

Answers (2)

ahmet alp balkan
ahmet alp balkan

Reputation: 45282

Assuming the underlying premise of this question is you're writing your own Kubernetes admission webhook, and trying to figure out concurrency model.

Kubernetes does not provide ordering or concurrency guarantees about when two different admission requests arrive to your webhook.

Imagine Req1 hitting Apiserver1 and a second later, Req2 hitting Apiserver2. If there's a slow network link between your webhook and Apiserver1, your webhook may see Req2 before Req1.

Even during no slow network links etc, your webhook (if it's written in Go etc) will handle each request concurrently by spawning a new goroutine.

As a result, you should not care about the concurrency model and assume each admission request is stateless and will arrive concurrently.

Therefore, it's not possible to write your admission webhooks in Kubernetes to do multi-resource constraints such as:

  1. Deny deletion of a Secret that's referenced by a Pod (because while the DELETE Secret request is allowed, a new Pod may be created referencing the Secret that's being deleted)

  2. Prevent two Pods that have the same image from being created (two admission requests can be validated at the same time concurrently, and won't see each other, and therefore both will be allowed)

Any such enforcement would be a "best effort" in webhooks.

Upvotes: 0

Andrew Skorkin
Andrew Skorkin

Reputation: 1376

Since in kube-api options we can see --max-mutating-requests-inflight and --max-requests-inflight flags, which are used to determine the server's total concurrency limit, I think that admission controller also support multi-threading. Because otherwise it will be a bottleneck for proceeding with API requests.

This is true and accurate advice, as long as we are using the base environment.

But on the other hand, we can customize our environment and specify how requests should be processed. For that purpose can be used API Priority and Fairness (APF). APF classifies and isolates requests in a more fine-grained way in comparison with --max-mutating-requests-inflight and --max-requests-inflight.

Without APF enabled, overall concurrency in the API server is limited by the kube-apiserver flags --max-requests-inflight and --max-mutating-requests-inflight. With APF enabled, the concurrency limits defined by these flags are summed and then the sum is divided up among a configurable set of priority levels. Each incoming request is assigned to a single priority level, and each priority level will only dispatch as many concurrent requests as its configuration allows.

The default configuration, for example, includes separate priority levels for leader-election requests, requests from built-in controllers, and requests from Pods.

So, it's a quite wide question. It all depends on which admission controller we using: original or custom, what controls are used (--max-mutating-requests-inflight and --max-requests-inflight command-line flags), APF configuration.

Upvotes: 0

Related Questions