CODEWITHSUNDEEP
google-cloud-platformterraformgoogle-cloud-iamworkload-identity
hb.Sara
hb.Sara

Reputation: 341

Terraform entreprise to GCP Workload Identity returninvalid_grant Error connecting to the given credential's issuer

I have set up a Workload identity Federation for an Terraform Entreprise installed on Azure using a GCP VM Agent.

From TFE I am facing this error

unable to generate access token: Post "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/{$SA}@{$projectid}.iam.gserviceaccount.com:
generateAccessToken": oauth2/google: status code 400: 
{"error":"invalid_grant","error_description":"Error connecting to the given credential's issuer."}

On The documentation explained that GCP must reach the OIDC metadata, As explained here

Both links seems reachable from the agent (VM on GCP used on TFE)

$ISSUER/.well-known/openid-configuration
$ISSUER/.well-known/jwks

Do you have any idea on what could the issue with my setup ?

Upvotes: 0

Views: 203

Answers (2)

hb.Sara
hb.Sara

Reputation: 341

Finally, I found a solution to my issue and I hope it will help other people. the issue here is that the two endpoints:

https://TFE_HOSTNAME/.well-known/openid-configuration
https://TFE_HOSTNAME/.well-known/jwks

should be publicly accessible as stated in the GCP documentation:

https://cloud.google.com/iam/docs/troubleshooting-workload-identity-federation#error-connecting-issuer

It was not the case in my example.

The Google recommandation following the documentation here, suggest to upload the jwks json file directly to the TFE provider,

OIDC provider with local JWKs To federate workloads that don't have a public OIDC endpoint, you can upload OIDC JSON Web Key Sets (JWKS) directly to the pool. This is common if you have Terraform or GitHub Enterprise hosted in your own environment or you have regulatory requirements not to expose public URLs. For more information, see Manage OIDC JWKs (Optional)

I don't know what the recommendations are for uploading the JWKS directly, but this is working!

Upvotes: 0

Sai Chandini Routhu
Sai Chandini Routhu

Reputation: 1

As per the Hashicorp Help Center troubleshooting article by Zachary Isom:

It is possible that your static OIDC metadata endpoints of the Terraform Enterprise installations are not accessible to the public, causing GCP to fail in verifying signed JWTs it receives from Terraform Enterprise.

Ensure that there is a way for GCP to communicate with these two endpoints:

https://TFE_HOSTNAME/.well-known/openid-configuration

https://TFE_HOSTNAME/.well-known/jwks

NOTE:

  • Check the network connectivity between the TFE VM on Azure and the GCP OIDC issuer endpoint https://iamcredentials.googleapis.com firewalls or network policies might be blocking the connection.
  • Check and confirm that the TFE VM has internet access and can reach Google public end points. No firewall rules blocking outbound traffic from the GCP VM agent to the OIDC metadata endpoints.
  • Ensure that the actual OIDC configuration files in those directories are accessible and are not corrupted and ensure that you have provided correct service account details such as ProjectID, service account name for GCP service account related to TFE workload identity.

Refer to this official Terraform document on Dynamic Provider Credentials for more information.

Upvotes: 1

Related Questions