Permissions issues with App engine apps and Datastore

Question

I'm trying to implement a simple design in google cloud using app engine standard and flexible with datastore. App1 lives in GAE standard environment. When a user interacts with this app, it writes some data to datastore and queues a task. The target of the queued task is App2 that lives in app engine flexible environment (the task can take a longer time to complete than standard environment allows). The idea is for App2 to read the data from datastore, perform the task using the data, once complete it should write a report entity to datastore. I've attached a simple diagram.

• In App1 I've set up a Service Account named flexKey with Owner permissions, downloaded the json file.

• when I run App2 locally I first export the path to the credentials json file as an environment variable:

  GOOGLE_APPLICATION_CREDENTIALS: "path/to/flexKey.json"

then launch the app with mvn jetty:run-exploded and everything works fine, App2 is able to authenticate with live datastore (not local emulation), and read the data written by App1. When I unset the environment variable, I get an 'Unauthenticated' error (expected)

• To use the same service account when App2 is deployed I've added the following in app.yaml for App2 to set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the service account json file flexKey.json (this is the path to the file on the deployed instance):

  env: flex
   env_variables: 
     GOOGLE_APPLICATION_CREDENTIALS: "/var/lib/jetty/webapps/root/WEB-INF/classes/flexKey.json"
   runtime: java
  

However, when I deploy App2 to app engine flexible environment, there is an error authenticating with datastore when trying to do the read query (this works fine when querying datastore with the same credentials from a locally running instance of App2):

com.google.cloud.datastore.DatastoreException: Missing or insufficient permissions.
    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:129)
    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:114)
    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.runQuery(HttpDatastoreRpc.java:182)
    at com.google.cloud.datastore.DatastoreImpl$1.call(DatastoreImpl.java:178)
    at com.google.cloud.datastore.DatastoreImpl$1.call(DatastoreImpl.java:174)
    at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
    at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
    at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
    at com.google.cloud.datastore.DatastoreImpl.runQuery(DatastoreImpl.java:173)

.....

the code is PERMISSION_DENIED

• If I leave out the environment variable GOOGLE_APPLICATION_CREDENTIALS from the app.yaml file for App2, then I get an 'Unauthenticated' error, so I think it's reading the file so I'm not sure what the issue is
• I'm using Objectify v6

I'm not able to see why the same credentials (created with Owner role) work fine when querying the datastore from a locally running instance of the app but don't work when datastore is queried from the deployed version of the same app (deployed to flexible environment). setting the path to the credentials file via an environment variable in app.yaml is the method recommended in the documentation unless I am mistaken.

• Is the GOOGLE_APPLICATION_CREDENTIALS environment variable not properly set in app.yaml?
• Is there something conceptually problematic about my design?

All help appreciated.

Answer 1

You have a blocking issue in your design: it is not possible for one application to enqueue tasks into a push queue targeted at a service from another application. From the <target> (push queues) rows in the Syntax tables for both queue.yaml and queue.xml references:

The string is prepended to the domain name of your app when constructing the HTTP request for a task. For example, if your app ID is my-app and you set the target to my-version.my-service, the URL hostname will be set to my-version.my-service.my-app.appspot.com.

If you want to use the task queue then you have to make the 2 services part of the same application. As a (positive) side effect you don't have to worry about setting up the authentication for datastore access anymore - both services can directly access the app's datastore.