CODEWITHSUNDEEP
amazon-web-servicesworkload-identity
kyuden
kyuden

Reputation: 409

GCP workload identity fails to execute Workflow

I want to execute a GCP Workflow with a workload identity (WI) for AWS.

I have done the following:

  1. create a WI pool for AWS with all default settings (no attribute condition specified)
  2. create a WI provider without any attribute conditions
  3. attach a service account with Workflow Invoker role and Workload Identity User role to the WI provider
  4. run the below command
gcloud workflows execute test \
--call-log-level=log-errors-only \
--location=asia-northeast1

This fails with

ERROR: (gcloud.workflows.execute) There was a problem refreshing your current auth tokens: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/meta-data/placement/availability-zone (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x104ca6520>: Failed to establish a new connection: [Errno 60] Operation timed out'))

What is the problem here? I have already confirmed that the service account itself can successfully run the command.

Upvotes: 0

Views: 481

Answers (1)

kyuden
kyuden

Reputation: 409

This happened because I was using my local terminal to execute this command. The error message says

HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/meta-data/placement/availability-zone

which means it cannot connect to the URL http://169.254.169.254/latest/meta-data/placement/availability-zone.

This in turn is the URL for EC2 instance metadata, which GCP uses to retrieve the region for that EC2 instance. This URL is only available within an EC2 instance, and attempting to connect to this URL from my terminal will cause a time out.

To fix this, you can do the following:

  • edit the credential JSON and remove lines for regionUrl and Url
  • export the following 2 environment variables: export AWS_ACCESS_KEY_ID=<your-access-key-id> and export AWS_REGION=<your-aws-region>

Now GCP will not use the instance metadata URL and you should stop seeing the timeout error.

Upvotes: 0

Related Questions