Google App Engine & Storage API auth error

Question

I have a web app that receives a file then attempts to add it to Google Cloud Storage, running on Google App Engine. This is the error I'm getting:

com.google.apphosting.api.ApiProxy$CallNotFoundException: The API package 'memcache' or call 'Get()' was not found.
at com.google.apphosting.api.ApiProxy$1.get(ApiProxy.java:173)
at com.google.apphosting.api.ApiProxy$1.get(ApiProxy.java:171)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:88)
at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:26)
at com.google.appengine.api.memcache.MemcacheServiceImpl.get(MemcacheServiceImpl.java:49)
at com.google.appengine.api.appidentity.AppIdentityServiceImpl.getAccessToken(AppIdentityServiceImpl.java:286)
at com.google.api.client.googleapis.extensions.appengine.auth.oauth2.AppIdentityCredential.intercept(AppIdentityCredential.java:98)
at com.google.api.client.googleapis.extensions.appengine.auth.oauth2.AppIdentityCredential$AppEngineCredentialWrapper.intercept(AppIdentityCredential.java:243)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:859)

This works on my LOCAL machine just fine (have gutil and everything setup). It fails with the above when deployed to GAE. Have checked the permissions, and it shows access to all Google Cloud API's. This code is the part that fails, directly from Google's sample at (https://github.com/GoogleCloudPlatform/java-docs-samples/blob/master/storage/json-api/src/main/java/StorageFactory.java)

 private static Storage buildService() throws IOException, GeneralSecurityException {
HttpTransport transport = GoogleNetHttpTransport.newTrustedTransport();
JsonFactory jsonFactory = new JacksonFactory();
GoogleCredential credential = GoogleCredential.getApplicationDefault(transport, jsonFactory);

if (credential.createScopedRequired()) {
  Collection<String> bigqueryScopes = StorageScopes.all();
  credential = credential.createScoped(bigqueryScopes);
}

return new Storage.Builder(transport, jsonFactory, credential)
    .setApplicationName("GCS Samples")
    .build();
}

For completeness, here's the code that uses the buildService above (again from the github sample link above)

            Storage client = StorageFactory.getService();
        Storage.Objects.Insert insertRequest = client.objects().insert(
                bucketName, objectMetadata, contentStream);
        insertRequest.execute();

The deps sections of pom.xml

  <parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.3.2.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependencies>
 <dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-web</artifactId>
</dependency>

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-test</artifactId>
  <scope>test</scope>
</dependency>

<dependency>
  <groupId>com.google.appengine.tools</groupId>
  <artifactId>appengine-gcs-client</artifactId>
  <version>RELEASE</version>
</dependency>

The GAE console shows it's using GAE v1.9.35. Here's the redacted info from an instance showing the perms available at the bottom

Module & version
Module: default
Version: 20160405t155800
App Engine version: 1.9.35
Managed by
Google
Availability
...
GAE managed VM for module: default, version: 20160405t155800
Tags
None
Machine type

g1-small (1 vCPU, 1.7 GB memory)
CPU platform

Intel Haswell
Zone

us-central1-c

IP forwarding

off
Boot disk and local disks

Name    Size (GB)   Type    Mode    
redacted
10  
Standard persistent disk
Boot, read/write

Delete boot disk when instance is deleted
Additional disks


Preemptibility  
Off (recommended)
Automatic restart   
Off
On host maintenance 
Terminate VM instance
Service account
-----
Cloud API access scopes

This instance has full API access to all Google Cloud services.    

Again, the app works locally and sends file uploads to Google Cloud Storage. Tried several times to upload the app to GAE without any luck. This app was deployed to the GAE Flexible editions, it shows this:

latest: Pulling from google_appengine/openjdk8

Also, using the GcsService lib fails in a different way, but I suspect it is because, for some reason, my app is not recognized properly on GAE as a GAE app with the proper authorization:

Caused by: java.io.IOException: java.lang.NullPointerException
at com.google.appengine.tools.cloudstorage.dev.LocalRawGcsService$BlobStorageAdapter.getInstance(LocalRawGcsService.java:186)
at com.google.appengine.tools.cloudstorage.dev.LocalRawGcsService$BlobStorageAdapter.access$000(LocalRawGcsService.java:109)
at com.google.appengine.tools.cloudstorage.dev.LocalRawGcsService.ensureInitialized(LocalRawGcsService.java:194)
at com.google.appengine.tools.cloudstorage.dev.LocalRawGcsService.beginObjectCreation(LocalRawGcsService.java:249)
at com.google.appengine.tools.cloudstorage.dev.LocalRawGcsService.beginObjectCreation(LocalRawGcsService.java:92)
at com.google.appengine.tools.cloudstorage.GcsServiceImpl$1.call(GcsServiceImpl.java:74)
at com.google.appengine.tools.cloudstorage.GcsServiceImpl$1.call(GcsServiceImpl.java:70)

code:

                GcsOutputChannel outputChannel = gcsService.createOrReplace(fileName, GcsFileOptions.getDefaultInstance());
            InputStream inputStream = uploadfile.getInputStream();
            copy(inputStream, Channels.newOutputStream(outputChannel));

Answer 1

You can use the GAE standard APIs (like in this example com.google.appengine.api.memcache) only on the base runtimes that support it. The list is there: https://cloud.google.com/appengine/docs/flexible/custom-runtimes/build#base_images

The base runtime used in your case is google_appengine/openjdk8 which does not support the GAE standard environment APIs.

Now, you can use the new Java Cloud APIs (this includes GCS storage access) via this gcloud-java library: https://github.com/GoogleCloudPlatform/gcloud-java