Kubernetes: API group and resources, what are their relations?

Question

When an application needs to call events API to get all events of its cluster, as a programmer I may define a role like this:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["events"]
  verbs: ["list"]

What makes me confusing is the apiGroups part, I can use "events.k8s.io", or simply "", or "events.k8s.io" and "" both...

What is this apiGroups thing? I tried to read official documentation but all I found is this:

API groups make it easier to extend the Kubernetes API. The API group is specified in a REST path and in the apiVersion field of a serialized object.

There are several API groups in Kubernetes:

The core (also called legacy) group is found at REST path /api/v1. The core group is not specified as part of the apiVersion field, for example, apiVersion: v1.

The named groups are at REST path /apis/$GROUP_NAME/$VERSION and use apiVersion: $GROUP_NAME/$VERSION (for example, apiVersion: batch/v1). You can find the full list of supported API groups in the Kubernetes API reference.

This doesn't help me understand it... Why there are named groups and core groups, why I can use "" and "events.k8s.io" together?

If my resource is events, why do I need to explicitly tell K8s that there is an API group named "events.k8s.io" as if the events in "events.k8s.io" and events in resources are two different things... This question had been haunting me for days😭

Answer 1

It has to do with Kubernetes controllers & CustomResourceDefinition.

When you write an operator for Kubernetes, you would define custom objects. For example, Elastic (www.elastic.co) has an operator deploying ElasticSearch/Kibana. They decided to call one of their object "elasticsearch".

Those resource names are arbitrary, and nothing guarantee there won't be someone else, working on its own operator, using the same names. Thus, when you define your custom resources, you would also affect them with an API group.

The API group is also arbitrary, though would usually identify a unique controller/operator/set of functionality. In the case of the ElasticSearch operator, they went with an API group "k8s.elastic.co". When installing this operator CRD on your cluster, a kubectl get crd would list objects such as "elasticsearchs.k8s.elastic.co", "kibanas.k8s.elastic.co".

You could very well deploy another operator, which implements its own "elasticsearch" or "kibana" objects, within its own API group.

Thus, when you write your Roles / ClusterRoles, that apiGroup field is crucial configuring your RBAC. The resources array lists short names, the apiGroup is then used resolving the fully qualified CRD name.

While kubernetes legacy objects use some empty string, other native objects may use networking.k8s.io, batch, extensions, ... In the meantime, custom controllers would all use their own api group, whenever they rely on their own CRDs.

---

Now, I'm not sure I know the answer to 'why I can use "" and "events.k8s.io" together?'. I would say it's historic?

I did not see any cases where setting multiple api groups in a rule was benefic. In my opinion, it's misleading, a source of mistake, ... when you find a rule with two api groups, you can be pretty sure the person who wrote it did not understand what they were doing.

At best, one combination of apigroup / resource grants what you meant to. Worst case, one combination of apigroup / resource grants something you did not want to allow in the first place.

Erratum: events are an exception, though. As pointed out in some kubernetes GitHub Issue ( https://github.com/kubernetes/kubernetes/issues/94857 ), some member would argue this is not a bug: we really need both API groups. Which is kind of surprising.

Not sure why would that be. Sidenote, the schema for Event in core/v1 and events.k8s.io isn't the same:

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#event-v1-events-k8s-io https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#event-v1-core

So they're not the same object strictly speaking, although from what I can see, they return the same data / in terms of etcd, it's the same data, with two JSON representations depending on which api group you query:

$ k get event.events.k8s.io -n kube-system
LAST SEEN   TYPE      REASON      OBJECT                                MESSAGE
58m         Warning   Unhealthy   pod/metrics-server-7df46996d8-c5wf5   Readiness probe failed: Get "https://10.233.108.212:8443/healthz": context deadline exceeded
...

$ k get event -n kube-system
LAST SEEN   TYPE      REASON      OBJECT                                MESSAGE
58m         Warning   Unhealthy   pod/metrics-server-7df46996d8-c5wf5   Readiness probe failed: Get "https://10.233.108.212:8443/healthz": context deadline exceeded
...

Not sure what's the rational here. events.k8s.io isn't anything new, I haven't read about some transition from one to the other ... That's a good question you have.

Answer 2

Why there are named groups and core groups

The reason is historical. Kubernetes first considered some resources part of a core group, meaning the core of Kubernetes is constituted by these resources (resources in the core group includes Pods, Events).

The purposes of the named API groups (or namespaces) (such as events.k8s.io) is to group resources by thematic. You'll have networking.k8s.io for resources related to networking.
The "core" resources were considered such an integral part of Kubernetes that they did not need a group (I'm simplifying).

why I can use "" and "events.k8s.io" together?

If my resource is events, why do I need to explicitly tell K8s that there is an API group named "events.k8s.io" as if the events in "events.k8s.io" and events in resources are two different things...

The API is not exactly the same. In the API reference you can see that in the events.k8s.io you can see some field staring with deprecated which are deprecated field(s) assuring backward compatibility with core.v1 Event type. So depending on which endpoint you use, the API semantic is not exactly the same (but Kubernetes has mechanisms to translate from one version to the other). Since some tools might rely on the core v1 version and not yet have migrated to events.k8s.io/v1, both need to exists.

There has been other "group migration" of that sort, notably from the extensions group to other group (Ingress migrated to networking.k8s.io, Deployment to apps)