CODEWITHSUNDEEP
google-app-enginegoogle-cloud-platformgoogle-kubernetes-enginenatgoogle-cloud-networking
Delta
Delta

Reputation: 157

IP address on GKE for egress to GAE appliction is 0.0.0.0, not IPv4 wth Cloud NAT

Long time passed after I posted a related issue. Link

Now, the issue has changed. With same procedure to make a cluster and NAT setting, GAE application shows 0.0.0.0 , not IPv6 address.

※GAE application is using a customized-domain.

In case that the egress communication is with on-prem server, the server shows a static IP address, as set by NAT.

Is there any change in a Google Internal networking? I would like to know how to avoid this issue, when I want an exact static IP address.

1.create a VPC

gcloud compute networks create ${vpc_name}\
--project=${project_id}\
--bgp-routing-mode=regional\
--subnet-mode=custom
2. create a subnet

gcloud compute networks subnets create ${subnet_name}\
--project=${project_id}\
--region=${region}\
--network=${vpc_name}\
--range=192.168.0.0/16
3. create an external static IP

gcloud compute addresses create ${external_ip_name}\
--region=${region}
4. create a route

gcloud compute routers create ${router_name}\
--region=${region}\
--network=${vpc_name}\
--asn=65001
5. create a nat

gcloud compute routers nats create ${nat_name}\
--region=${region}\
--router=${router_name}\
--nat-external-ip-pool="${external_ip}"\
--nat-all-subnet-ip-ranges

6. make a cluster

gcloud container clusters create ${gke_cluster_name}\
--project=${project_id}\
--zone=${zone}\
--network=${vpc_name}\
--subnetwork=${subnet_name}\
--enable-ip-alias\
--enable-private-nodes\
--master-ipv4-cidr=172.16.0.0/28\
--enable-master-authorized-networks\
--master-authorized-networks=0.0.0.0/0\
--no-enable-legacy-authorization\
--no-enable-basic-auth\
--no-issue-client-certificate\
--num-nodes=${NUM_NODE}\
--enable-autoscaling --max-nodes=${MAX_NUM} --min-nodes=${MIN_NUM}\
--scopes="cloud-platform"

7. deploy an application to the cluster and confirm the IP

Make a https-connection from the application to an other gcp-project's application. And detect the IP address of origin-request as 'REMOTE_ADDR'.

Upvotes: 4

Views: 1847

Answers (1)

Raynel A.S
Raynel A.S

Reputation: 485

Thanks for your clarification. So basically this is what we have:

Private GKE -> Cloud NAT -> External GAE load balancer -> GAE

and the App in GAE is seeing an ipv6 address

When you use Cloud NAT, Google Private Access is enabled by default and so requests from your VPC to GAE will use Private Access.

Private Access will use the Google internal network to route the request to ensure the public internet is not used. To properly route this, we use IPv6.

To explain the IPv6 address:

Google uses a set NAT64 prefix: fda3:e722:ac3:10::/64. This value is constant across all GCP VPC networks. The last 32 bits are converted directly from the internal IP address of the host making the request. This is converted from binary to hex (ie 10.132.0.5 in decimal -> 0x0a840005 in hex).

The middle part between the NAT64 prefix and the last 32 bits is a unique identifier for your VPC. This identifier is unique to each VPC network, thus you can have more than 1 per project.

Currently, there is no way to predict this identifier beforehand. The best way to get this information is by sending a request to GAE through Private Access and take note of the IPv6 listed as the source. The IPv6 address is available in the response from the server, and in the Stackdriver Logs.

Using this information, you should be able to adequately maintain your whitelist for your GAE hosted applications.

All that being said, if you are seeing The IPv4 addresses from the GKE pods show up as 0.0.0.0 in the GAE application. I encourage you to report this using this link

Upvotes: 3

Related Questions