GCP Firestore Python Credentials

Question

I am having trouble sending data from my Linux VM to my GCP's Firestore. I am simply trying to update an item inside of the database. I am getting an issue regarding credentials. Depending on the method I use, I get different errors however, I believe they all stem from the same issue.

As a note, I have a service account, with the json credentials. I know that these credentials are still valid because I am able to use GCP's Speech-to-Text. The only strange behavior I am getting is when accessing the Firestore.

Note: I am using the Firestore from the GCP console rather than the Firebase console. I see that there are slight differences in how they are used (mostly security stuff).

If I simply open a new terminal and run my python script (I do not set any reference to my credentials file). The data is successfully added to the database, but I get the following warning:

UserWarning: Your application has authenticated using end user credentials from Google Cloud 
SDK. We recommend that most server applications use service accounts instead. If your 
application continues to use end user credentials from Cloud SDK, you might receive a "quota 
exceeded" or "API not enabled" error. For more information about service accounts, see 
https://cloud.google.com/docs/authentication/

On the other hand, if I run the line export GOOGLE_APPLICATION_CREDENTIALS="folder/file.json" before running the same python script. I will get this error instead:

Traceback (most recent call last):
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/home/vagrant/.local/lib/python3.6/site-packages/grpc/_channel.py", line 565, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/home/vagrant/.local/lib/python3.6/site-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
    status = StatusCode.PERMISSION_DENIED
    details = "Missing or insufficient permissions."
    debug_error_string = "{"created":"@653.64","description":"Error received 
from peer ipv4:...:","file":"src/core/lib/surface
/call.cc","file_line":1052,"grpc_message":"Missing or insufficient 
permissions.","grpc_status":7}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "sendToDB.py", line 28, in <module>
    update_create_if_missing(args.uid, args.words)
  File "sendToDB.py", line 16, in update_create_if_missing
    item.update({'text': firestore.ArrayUnion([words])})
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/cloud/firestore_v1/document.py", line 382, in update
    write_results = batch.commit()
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/cloud/firestore_v1/batch.py", line 147, in commit
    metadata=self._client._rpc_metadata,
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/cloud/firestore_v1/gapic/firestore_client.py", line 1042, in commit
    request, retry=retry, timeout=timeout, metadata=metadata
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/api_core/gapic_v1/method.py", line 143, in __call__
    return wrapped_func(*args, **kwargs)
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/api_core/retry.py", line 273, in retry_wrapped_func
    on_error=on_error,
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/api_core/retry.py", line 182, in retry_target
    return target()
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout
    return func(*args, **kwargs)
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.PermissionDenied: 403 Missing or insufficient permissions.

If I state the the credentials file directly in the python file using a line similar to this db = firestore.Client(project="myproj-99999",credentials="folder/file.json") I get this error:

Traceback (most recent call last):
  File "sendToDB.py", line 23, in <module>
    update_create_if_missing(args.uid, args.words)
  File "sendToDB.py", line 9, in update_create_if_missing
    db = firestore.Client(project="earningstotext-251320",credentials="keys/gcpcmdlineuser.json")
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/cloud/firestore_v1/client.py", line 105, in __init__
    project=project, credentials=credentials, _http=None
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/cloud/client.py", line 227, in __init__
    Client.__init__(self, credentials=credentials, _http=_http)
  File "/home/vagrant/.local/lib/python3.6/site-packages/google/cloud/client.py", line 130, in __init__
    raise ValueError(_GOOGLE_AUTH_CREDENTIALS_HELP)
ValueError: This library only supports credentials from google-auth-library-python. 
See https://google-cloud-python.readthedocs.io/en/latest/core/auth.html 
for help on authentication with this library.

Note here the readthedocs link here is broken.

Essentially, I would just like to be able to update and read from my Firestore using python without any errors or warnings. I also can't make sense why, the one time it does work (when I get the warning) is when I don't set any sort of credentials.

This is a similar question to my previous question found here. However I think my new information warranted a seperate question.

Answer 1

You need to add the "Cloud Datastore Owner" role to the service account you are using.

1. Go to IAM & admin> IAM section of console.cloud.google.com
2. Look for the service account you are using and click the pencil icon to edit it.
3. Click Add Another Role, and search for datastore owner.

Ref: https://cloud.google.com/firestore/docs/security/iam