CODEWITHSUNDEEP
pythongoogle-cloud-platformgoogle-cloud-pubsubgoogle-authenticationgoogle-api-python-client
istrupin
istrupin

Reputation: 1533

Using ImpersonatedCredentials with python api for GCP PubSub SubscriberClient

I'm attempting to create a SubscriberClient using ImpersontatedCredentials (impersonating a service account) using the python pubsub API, and then creating some subscriptions with that client. I'm attempting to try to run this code locally (though eventually it will get deployed as a cloud function)

In doing so, I get this error:

filter: "attributes.provider_id = "integration_test_create_delete_provider_id_123"" , metadata=[('x-goog-request-params', 'name=projects/my-project/subscriptions/integration_test_create_delete_provider_id_123'), ('x-goog-api-client', 'gl-python/3.8.6 grpc/1.39.0 gax/1.31.1 gccl/2.7.0')]), last exception: 503 Getting metadata from plugin failed with error: ('Unable to acquire impersonated credentials: No access token or invalid expiration in response.', '{\n "error": {\n "code": 400,\n "message": "Request contains an invalid argument.",\n "status": "INVALID_ARGUMENT"\n }\n}\n')

My code to create the impersonated credentials looks like this:

from google.auth import impersonated_credentials
from google.cloud import pubsub_v1

target_scopes = [
"https://www.googleapis.com/auth/cloud-platform",
"https://www.googleapis.com/auth/pubsub",
]
target_principal = ("[email protected]",)

def _get_impersonated_pubsub_client():
    return pubsub_v1.SubscriberClient(credentials=_get_impersonated_creds())


def _get_impersonated_creds():
    default_creds, _ = google.auth.default()
    impersonated_creds = impersonated_credentials.Credentials(
        source_credentials=default_creds,
        target_principal=target_principal,
        target_scopes=target_scopes,
    )
    return impersonated_creds

From there, I call create_subscription(request=my_request) on the client I get from calling _get_impersonated_pubsub_client(), and after about a minute I see the above error.

My request object looks like this:

{
        "name": "projects/my-project/subscriptions/integration_test_create_delete_provider_id_123",
        "topic": "projects/my-project/topics/my-topic-dev-us-west1",
}

If I make the same call without impersonation or specifying credentials in the SubscriberClient constructor, everything works great.

My immediate first thought was it was a permission error, but I double checked that my source account has the ServiceAccountTokenCreator role.

Additionally, I tried to run an equivalent command with the GCloud CLI (including the impersonation) like this:

gcloud pubsub subscriptions create my_subscription_name --topic=projects/my-project/topics/my-topic-dev-us-west1 --impersonate-service-account [email protected]

This works exactly as expected -- which leads me to believe the permissions in GCP are set up correctly.

Does the SubscriberClient not work with ImpersonatedCredentials, or is there some coercion I could do to make the auth work?

I also made sure to run gcloud config unset auth/impersonate_service_account and then gcloud auth application-default login to ensure my local credentials were in a good state.

Relevant dependencies:

google-api-core[grpc]==1.31.1; platform_python_implementation != 'PyPy'
google-api-python-client==2.15.0; python_version >= '3.6'
google-auth-httplib2==0.1.0
google-auth==1.34.0
google-cloud-core==2.0.0b1; python_version >= '3.6'
google-cloud-firestore==2.2.0; platform_python_implementation != 'PyPy'
google-cloud-pubsub==2.7.0
google-cloud-storage==1.41.1; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'
google-crc32c==1.1.2; python_version >= '3.6'
google-resumable-media==2.0.0b1; python_version >= '3.6'
googleapis-common-protos[grpc]==1.53.0; python_version >= '3.6'
grpc-google-iam-v1==0.12.3
grpcio==1.39.0
httplib2==0.19.1

Upvotes: 3

Views: 4661

Answers (1)

John Hanley
John Hanley

Reputation: 81336

IMHO your Python code appears correct.

Complete the following steps:

  1. List the services that are enabled:

gcloud services list --enabled

  1. Verify that iamcredentials.googleapis.com is enabled. To enable:

gcloud services enable iamcredentials.googleapis.com

  1. Verify that cloudresourcemanager.googleapis.com is enabled. To enable:

gcloud services enable cloudresourcemanager.googleapis.com

  1. Verify that the Compute Engine default service account has the role roles/iam.serviceAccountTokenCreator or better.

The required permission are:

  • iam.serviceAccounts.getAccessToken

  • iam.serviceAccounts.getOpenIdToken (this permission is required to fetch identity tokens).

    gcloud projects add-iam-policy-binding [PROECT_ID]
    --member "serviceAccount:[GCE_DEFAULT_SA_FULL_EMAIL]"
    --role "roles/iam.serviceAccountTokenCreator"

  1. Verify that the Compute Engine default service account has the role roles/serviceusage.serviceUsageConsumer or better.

    gcloud projects add-iam-policy-binding [PROECT_ID]
    --member "serviceAccount:[GCE_DEFAULT_SA_FULL_EMAIL]"
    --role "roles/serviceusage.serviceUsageConsumer"

If any of the previous are not enabled/granted, token issuance will fail.

Note: If you only want the Compute Engine service account to be able to impersonate one service account, change step 4 to grant the role on the service account instead of the project:

gcloud iam service-accounts add-iam-policy-binding [SA_FULL_EMAIL] \
--member serviceAccount:[GCE_DEFAULT_SA_FULL_EMAIL] \
--role roles/iam.serviceAccountTokenCreator

Upvotes: 2

Related Questions