What is the mechanism of Terraform state locking when using Google Cloud Platform?

Question

What is the Google Cloud Platform mechanism for locking state file when using Terraform? Something like DynamoDB on AWS...

thanks

Answer 1

During a Terraform operation, Terraform creates a lock file autometically with a .tflock extension in the same GCS bucket and prefix as the state file. For example, if the state file is located at gs://my-bucket/terraform/state.tfstate, the lock file would be at gs://my-bucket/terraform/state.tflock.

Terraform uses the DoesNotExist precondition of GCS to check if the lock file already exists. This is implemented using the HTTP header x-goog-if-generation-match: 0. If the lock file does not exist (generation = 0), the lock operation succeeds. If the lock file exists, the operation fails, indicating that another process is currently holding the lock.

Once the Terraform operation is complete, the lock file is deleted. This release of the lock allows other processes to acquire it for their operations.

Answer 2

gcs backend implements Terraform state locking by using a special lock file with .tflock extension. This file is placed next to the Terraform state itself for the period of Terraform state operation. For example, if the state file is located at path

gs://BUCKET/PREFIX/WORKSPACE.tfstate

then the corresponding lock file will be located at path

gs://BUCKET/PREFIX/WORKSPACE.tflock

Source: hashicorp/terraform

The atomicity of locking is guaranteed by using the GCS feature called Precondition. Terraform itself makes use of DoesNotExist condition of GCP Go SDK which in turn uses the GCS Precondition. Underneath, this adds this HTTP header x-goog-if-generation-match: 0 to the GCS copy request.

According to GCS documentation:

When a Match precondition uses the value 0 instead of a generation number, the request only succeeds if there are no live objects in the Cloud Storage bucket with the name specified in the request.

Which is exactly what is needed for Terraform state locking.

Answer 3

Google Cloud Platform like most of the remote backends natively supports locking. AWS doesn't support locking natively via S3 but it does as you mentioned via DynamoDB.

To run terraform apply, Terraform will automatically acquire a lock; if someone else is already running apply, they will already have the lock, and you will have to wait.

You can run apply with the -lock-timeout=<TIME> parameter to tell Terraform to wait up to TIME for a lock to be released (e.g., -lock-timeout=10m will wait for 10 minutes).

Answer 4

Where you store the state files (defined using a backend) is distinct from where you're deploying to. They could be the same, but don't have to be. For example, you could deploy resources to Azure while storing the state file in an AWS S3 bucket.

If you're interested in storing the state file in the Google Cloud, Terraform has a backend called gcs that includes locking. To quote the documentation:

gcs stores the state as an object in a configurable prefix and bucket on Google Cloud Storage (GCS).