CODEWITHSUNDEEP
google-cloud-platformnetwork-programmingvpc
piotrekkr
piotrekkr

Reputation: 3181

Failed to release serverless ipv4 because in use by some address reservation

For some time now I try to destroy GCP project using terraform destroy. Almost everything was destroyed but I had some errors saying that I cannot remove

module.transformers.google_service_networking_connection.private_vpc_connection: Destroying... [id=projects%2F{PROJECT}%2Fglobal%2Fnetworks%2Fprivate-network:servicenetworking.googleapis.com]
module.transformers.google_compute_subnetwork.direct_vpc_access_cloud_run: Destroying... [id=projects/{PROJECT}/regions/europe-west1/subnetworks/direct-vpc-access-cr]
module.transformers.google_service_networking_connection.private_vpc_connection: Still destroying... [id=projects%2F{PROJECT}...twork:servicenetworking.googleapis.com, 10s elapsed]
module.transformers.google_service_networking_connection.private_vpc_connection: Still destroying... [id=projects%2F{PROJECT}...twork:servicenetworking.googleapis.com, 20s elapsed]
╷
│ Error: Error when reading or editing Subnetwork: googleapi: Error 400: The subnetwork resource 'projects/{PROJECT}/regions/europe-west1/subnetworks/direct-vpc-access-cr' is already being used by 'projects/{PROJECT}/regions/europe-west1/addresses/serverless-ipv4-{IPV4_ID}', resourceInUseByAnotherResource
│ 
│ 
╵
╷
│ Error: Unable to remove Service Networking Connection, err: Error waiting for Delete Service Networking Connection: Error code 9, message: Failed to delete connection; Producer services (e.g. CloudSQL, Cloud Memstore, etc.) are still using this connection.
│ Help Token: XXXXX
│ 
│ 
╵

I managed to manually remove module.transformers.google_service_networking_connection.private_vpc_connection in GCP Console but this TF error remains.

When trying to remove IPv4 address manually in GCP Console I get this error:

Failed to release serverless-ipv4-{IPV4_ID}: The address resource 'projects/{PROJECT_ID}/locations/europe-west1/addresses/serverless-ipv4-{IPV4_ID}' is already being used by '//serverless.googleapis.com/projects/{PROJECT_ID}/locations/europe-west1/addressReservations/serverless-ipv4-{IPV4_ID}'

It is basically saying that IPv4 with IPV4_ID cannot be removed because there is some "reservation" using it. But I cannot find any "reservations" in GCP Cloud Console. I also tried to disable Cloud Run API and Service Networking APIs but still I cannot remove this IP and drop fully my project.

Any idea how can I remove this IPv4 address? Thank you.

Upvotes: 4

Views: 1265

Answers (2)

Felix Seifert
Felix Seifert

Reputation: 602

The GCP docs describe it aptly: After you delete or move your Cloud Run resources, wait 1-2 hours for Cloud Run to release the IPs before you delete the subnet.

The relevant Cloud Run PM knows about this issue and they have an internal feature request for it. The current solution is simply to wait before releasing the IP.

Depending on your situation, you might be interested to introduce a sleep dependency in between destroying the resources. This is not a great workaround because the destroy would take over an hour, but it would make the destroy succeed.

Alternatively, it could be an option to simply make Terraform "forget" about the resource if the resource would be removed automatically anyway.

Upvotes: 3

piotrekkr
piotrekkr

Reputation: 3181

Finally found a fix.

I was looking at IP address and figured out that this was probably not from Cloud Run. It was different subnet. Only one more service in project was connected to VPC and it was Cloud SQL. I was connecting SQL instance with private IP using service networking connection. This automatically created IP address to use inside VPC for this connection. Somehow after you drop SQL instance and even disable service networking API, IP and "reservation" is still there in project preventing removal of subnets and VPC.

I disabled Cloud SQL API and waited like minute. Then tried manually remove IP address again. Now it worked without any error.

Still I don't know how to remove this "reservation" or where to find it. Somehow disabling whole Cloud SQL API removed it.

Hope it helps someone.

// EDIT

Had similar issue with other project but this time I did not disable any APIs. After like half a day I tried again and this time it was removed without issues. Seems like those IPs are reserved for at least few hours before they can be removed. If you are not in a hurry consider retrying next day.

Upvotes: 3

Related Questions