Set up Google Cloud Platform (GCP) authentication for Terraform Cloud

Question

How do I configure Terraform Cloud to authenticate with Google Cloud Platform (GCP)?

Right now, my configuration is looking like this:

provider "google" {
  project = "my-project-id"
  region = "europe-west3"
  zone = "europe-west3-a"
}

resource "google_storage_bucket" "my-bucket" {
  name = "my-bucket"
  location = "EUROPE-WEST3"
  force_destroy = true
  uniform_bucket_level_access = true
}

And this gets me an error:

Error: Attempted to load application default credentials since neither credentials nor access_token was set in the provider block. No credentials loaded. To use your gcloud credentials, run 'gcloud auth application-default login'. Original error: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

Answer 1

You can use variable sets in terraform cloud as well.

1. Create a variable set (you can find it in the setting when you are in your organization in Terraform Cloud)
2. Name the variable set GOOGLE_CREDENTIALS
3. Create a variable inside this variable set the Key to GOOGLE_CREDENTIALS
4. Copy the contents of the access token JSON file, and remove all the spaces and new lines (I used the command cat google_access_token_file.json | sed ':a;N;$!ba;s/\n/ /g' | sed 's/[[:space:]]//g') and save the output in the Value field
5. Make sure you are allowing the variable set to be applied on the required workspaces (if you don't want to use it across all workspaces in your organization) or select "Apply globally" so that the variable set can be applied to all the workspaces in your organization
6. Select the environment variable radio button and check the sensitive field (recommended) as shown in the screenshot
7. Save the variable set and that's it! Your variable set and the variables within the set will be used by Terraform Cloud without you having to configure the credential variables in your templates

Answer 2

As it is written in the error message, you need to login with your account with: gcloud auth application-default login. I had a problem that even logging in didn't help.

There is one simple solution if you want to use your personal account, like I did when I encountered this problem. I didn't want to create a Service Account yet, but I do agree that Service Account is a way to go if you're not just playing around.

You can pass your access token to the provider in the following way:

provider "google" {
  access_token = file("./${var.google_access_token_file}")
}

As you can notice, I kept that token in a file that I kept locally. You can get the access token with the following command:

export TF_VAR_google_access_token=$(gcloud auth print-access-token)

Answer 3

When you have created a service account, you will get an option to download the JSON file. The file will contain the below information.

{
  "type": "service_account",
  "project_id": "aaaa-350310",
  "private_key_id": "28e8c8ssssssdf43e6d3849a4202642a8a0cd9cd5c2696",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMI...\n-----END PRIVATE KEY-----\n",
  "client_email": "ssss@sss-350310.iam.gserviceaccount.com",
  "client_id": "sss",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/fetebird%40fetebird-350310.iam.gserviceaccount.com"
}

You need to pass file as an environment variable

On Mac

export GOOGLE_APPLICATION_CREDENTIALS="download_location/file_name.json"

Answer 4

First of all, you will need to set up a service account in your GCP project in order for Terraform Cloud to be able to manage resources for you. Just do the following:

1. Log in to the GCP console and switch to the desired project.
2. Go to the IAM & Admin → Service accounts section.
3. Press the "Create service account" button.
4. Specify a meaningful name for your service account and click "Create and continue".
5. Specify a role for your service account. For test purposes you can use the Owner role with the maximum permissions. However, in production I would highly recommend to create a separate role for your service account with minimal possible permissions.
6. Then click "Done" to finally create the service account.
7. Now, select the newly created account from the list and go to the "KEYS" tab.
8. Press the "ADD KEY" button and select the "Create a new key" option.
9. Select the JSON format and press "CREATE".
10. Download the key file to your machine and open it in your favorite text editor.
11. The provided key is in multiline JSON format, however, in order to be able to use it in Terraform configuration it should be minified. You can use any JSON minifier that you can trust. Otherwise, you can use a "find & replace" functionality of your text editor to remove all multiline characters. In the end you should receive a JSON document as a single line of text, copy it.

Now, you will need to specify the JSON key in your Terraform configuration. The most straightforward way to do so would be to put it directly in your google provider configuration under the credentials property. However, it is a VERY BAD practice to store such sensitive data in your code. We would do something else instead:

1. Add the following variable declaration to your Terraform configuration file:

variable "gcp_credentials" {
  type = string
  sensitive = true
  description = "Google Cloud service account credentials"
}

This will tell Terraform that this input variable actually exists and could be used to configure the stack.

2. Then, go to your Terraform Cloud console and switch to the desired workspace. Go to the "Variables" tab.

3. Now, press the "Add variable" button and specify the following data:

   • Key: gcp_credentials
   • Value: INSERT YOUR SINGLE-LINE JSON HERE
   • Description: Google Cloud service account credentials
   • Check the "Sensitive" checkbox.

4. Click the "Save variable button".

Finally, update your provider configuration to look like this:

provider "google" {
  project = "my-project-id"
  credentials = var.gcp_credentials
  region = "europe-west3"
  zone = "europe-west3-a"
}

Using this approach, your secret JSON key would be securely stored by the Terraform Cloud without the ability for anybody to read it directly (thanks to the "sensitive" option) and the key would be provided to the Google Provider at runtime by the means of Terraform input variable.

However, be advised that it's not a bullet-proof way of storing secrets and there are situation when the content of the variable could be read by a party with the ability to update the configuration and read the log files.

---

I would also recommend to move other information such as region/zone and project ID from the config file. This will make your stack more reusable and your configuration cleaner (by removing duplication).

Here's the final example:

#===========#
# VARIABLES #
#===========#

variable "gcp_project_id" {
  type = string
  description = "Google Cloud project ID"
}

variable "gcp_credentials" {
  type = string
  sensitive = true
  description = "Google Cloud service account credentials"
}

variable "gcp_region" {
  type = string
  description = "Google Cloud region"
}

variable "gcp_zone" {
  type = string
  description = "Google Cloud zone"
}


#===========#
# PROVIDERS #
#===========#

provider "google" {
  project = var.gcp_project_id
  credentials = var.gcp_credentials
  region = var.gcp_region
  zone = var.gcp_zone
}


#===========#
# RESOURCES #
#===========#

resource "google_storage_bucket" "my-bucket" {
  name = "my-bucket"
  location = var.gcp_region
  force_destroy = true
  uniform_bucket_level_access = true
}