CODEWITHSUNDEEP
google-kubernetes-enginekubernetes-networkpolicy
Tim Schwalbe
Tim Schwalbe

Reputation: 1736

Network Policy always blocks Metadata Server on GKE

I am using GKE and I want to write some network policies, but as soon as I add the following Policy I get this error:

url.Error Get "http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/scopes

When I delete the policy everything works.

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: {{ template "name" . }}
spec:
  podSelector:
    matchLabels:
      app: {{ template "name" . }}
  policyTypes:
  - Egress
  egress:
  - ports:
    - port: 53
      protocol: UDP
    - port: 53
      protocol: TCP
    - port: 443
      protocol: TCP
    - port: 3307
      protocol: TCP
    - port: 3306
      protocol: TCP
  - to:
    - ipBlock:
        cidr: 169.254.169.254/32

Upvotes: 2

Views: 1116

Answers (2)

Tim Schwalbe
Tim Schwalbe

Reputation: 1736

Here is the working solution:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: {{ template "name" . }}
spec:
  podSelector:
    matchLabels:
      app: {{ template "name" . }}
  policyTypes:
    - Egress
  egress:
    - ports:
      - port: 53
        protocol: UDP
      - port: 53
        protocol: TCP
      - port: 443
        protocol: TCP
      - port: 3307
        protocol: TCP
    - to:
      - ipBlock:
          cidr: 169.254.169.252/32
      ports:
        - protocol: TCP
          port: 988

Upvotes: 1

Hemanth Kumar
Hemanth Kumar

Reputation: 3762

If you use network policy with GKE Workload Identity, you must allow egress to the following IP addresses and port numbers so your Pods can communicate with the GKE metadata server. For clusters running GKE version 1.21.0-gke.1000 and later, allow egress to 169.254.169.252/32 on port 988. For clusters running GKE versions before 1.21.0-gke.1000, allow egress to 127.0.0.1/32 on port 988. To avoid disruptions during auto-upgrades, allow egress to all of these IP addresses and ports.

So, As per your YAML file, you need to pass the port 988 also to allow egress because cidr: 169.254.169.254/32 as this pass only on 988 port. So that there will not be any error on metadata.

An example NetworkPolicy might look like this:

Upvotes: 1

Related Questions