CODEWITHSUNDEEP
pythongoogle-cloud-storagegoogle-compute-enginegoogle-cloud-python
Matti
Matti

Reputation: 484

Blob.generate_signed_url() failing to AttributeError

So I'm trying to produce temporary globally readable URLs for my Google Cloud Storage objects using the google-cloud-storage Python library (https://googlecloudplatform.github.io/google-cloud-python/latest/storage/blobs.html) - more specifically the Blob.generate_signed_url() method. I doing this from within a Compute Engine instance in a command line Python script. And I keep getting the following error:

AttributeError: you need a private key to sign credentials.the credentials you are currently using <class 'oauth2cl
ient.service_account.ServiceAccountCredentials'> just contains a token. see https://google-cloud-python.readthedocs
.io/en/latest/core/auth.html?highlight=authentication#setting-up-a-service-account for more details.

I am aware that there are issues with doing this from within GCE (https://github.com/GoogleCloudPlatform/google-auth-library-python/issues/50) but I have created a new Service Account credentials following the instructions here: https://cloud.google.com/storage/docs/access-control/create-signed-urls-program and my key.json file most certainly includes a private key. Still I am seeing that error.

This is my code: 

keyfile = "/path/to/my/key.json"
credentials = ServiceAccountCredentials.from_json_keyfile_name(keyfile)
expiration = timedelta(3) # valid for 3 days
url = blob.generate_signed_url(expiration, method="GET",
                               credentials=credentials)

I've read through the issue tracker here https://github.com/GoogleCloudPlatform/google-cloud-python/issues?page=2&q=is%3Aissue+is%3Aopen and nothing related jumps out so I am assuming this should work. Cannot see what's going wrong here.

Upvotes: 27

Views: 18746

Answers (4)

amontero
amontero

Reputation: 79

You can run GCloud SDK outside of GCP using Application Default Credentials (as you do with your user account) and make it work as a Service Account by "impersonating the SA". Notice, though, that for this to work you need to have the "Service Account Token Creator" role, which will allow you to create tokens "impersonating" another account (the Service Account).

When you have the permission, run the gcloud auth ADC login procedure, but "upgrading" you ADC credentials to an impersonated SA, like this:

$ gcloud auth application-default login --impersonate-service-account=[your-service-account-id]@developer.gserviceaccount.com

After that, your local ADC credentials are tied to the said SA. Keep in mind that audit trails for operations performed with this SA keep record of who requested the impersonation (ie. you). You can check it by comparing your current ADC credentials file with the newly generated above.

This way, your code will run outside GCP exactly the same than on GCP, without any conditional code and without having to juggle with JSON credentials, which is highly discouraged by Google.

Upvotes: 2

Mito
Mito

Reputation: 151

Currently, it's not possible to use blob.generate_signed_url without explicitly referencing credentials. (Source: Class Blob - generate_signed_url) However, you can do a workaround, as seen here, which consists of:

signing_credentials = compute_engine.IDTokenCredentials(
    auth_request,
    "",
    service_account_email=credentials.service_account_email
)
signed_url = signed_blob_path.generate_signed_url(
    expires_at_ms,
    credentials=signing_credentials,
    version="v4"
)

Upvotes: 15

Chukwuma Nwaugha
Chukwuma Nwaugha

Reputation: 715

A much complete snippet for those asking where other elements come from. cc @AlbertoVitoriano

    from google.auth.transport import requests
    from google.auth import default, compute_engine
    
    credentials, _ = default()
    
    # then within your abstraction
    auth_request = requests.Request()
    credentials.refresh(auth_request)
    
    signing_credentials = compute_engine.IDTokenCredentials(
        auth_request,
        "",
        service_account_email=credentials.service_account_email
    )
    signed_url = signed_blob_path.generate_signed_url(
        expires_at_ms,
        credentials=signing_credentials,
        version="v4"
    )

Upvotes: 5

tomasn4a
tomasn4a

Reputation: 615

I was having the same issue. Ended up fixing it by starting the storage client directly from the service account json.

storage_client = storage.Client.from_service_account_json('path_to_service_account_key.json')

I know I'm late to the party but hopefully this helps!

Upvotes: 21

Related Questions