Getting a 401 Unauthorized while accessing appengine environments using remote api

Question

I am having some trouble accessing some appengine projects with remote api from my local machine. It fails with a 401 - urllib2.HTTPError: HTTP Error 401: Unauthorized Too many auth attempts.

I am using the following command to start the remote api shell: python /usr/lib/google-cloud-sdk/platform/google_appengine/remote_api_shell.py -s <version>-dot-<my appengine project>.appspot.com

Am using gcloud version 200.0.0. My credentials have been saved to ~/.config/gcloud/application_default_credentials.json after I ran gcloud auth application-default login and completed the web authentication flow. I have confirmed that the same code is deployed to all our appengine environments and the following is also set in app.yaml

builtins:
- remote_api: on

as per https://cloud.google.com/appengine/docs/standard/python/tools/remoteapi

We have quite a few appengine projects. This command works on all of them, but consistently fails on 2 of them. As per the IAM & admin page, my email is listed as an owner on one of the two environments where its failing(am thinking that it should at least work on that environment, but its not). Is there anything else I need to be doing to have remote access to an environment? or has anybody else has experienced this lately.

Attaching full stack trace below

Traceback (most recent call last):
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/remote_api_shell.py", line 133, in <module>
    run_file(__file__, globals())
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/remote_api_shell.py", line 129, in run_file
    execfile(_PATHS.script_file(script_name), globals_)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 160, in <module>
    main(sys.argv)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 156, in main
    oauth2=True)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 74, in remote_api_s
hell
    secure=secure, app_id=appid)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 1052, in Co
nfigureRemoteApiForOAuth
    rpc_server_factory=rpc_server_factory)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 1137, in Co
nfigureRemoteApi
    app_id = GetRemoteAppIdFromServer(server, path, rtok)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 842, in Get
RemoteAppIdFromServer
    response = server.Send(path, payload=None, **urlargs)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 259, in Send
    NeedAuth()
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 235, in NeedA
uth
    RaiseHttpError(url, response_info, response, 'Too many auth attempts.')
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 85, in RaiseH
ttpError
    raise urllib2.HTTPError(url, response_info.status, msg, response_info, stream)
urllib2.HTTPError: HTTP Error 401: Unauthorized Too many auth attempts.

Answer 1

I signed up for Google Cloud paid support to get a fix for this. It took a little while, but the solution is to add the --secure flag like this:

python remote_api_shell.py --secure app-name

I have no idea what changed behind the scenes so that this extra flag is now required. I haven't made any recent changes to my website or project that could have caused this.

It is also troubling that the default mode for remote_api_shell.py is insecure, but at least there is a solution.

Below is my original answer that is no longer needed.

---

I was able to get this working, but it is kind of a hack and I would like to get it fixed the right way.

Here is the hack:

• Create a new service account on the Cloud Console and make it a project owner
• Download the secret key as a JSON file
• export GOOGLE_APPLICATION_CREDENTIALS=[PATH TO JSON FILE]

Now remote_api_shell.py works.

It looks like one of previous service accounts got corrupted, but I have no idea how.

Answer 2

I'm not an expert on this, but the Application Default Credential appears to be linked to a user account rather than to an app's service account. (I assume this because I can connect to two of my apps using remote_api_shell.py with a single credential obtained from gcloud auth application-default login).

Is it possible that the user account that you're using isn't listed as an owner or editor on that apps that aren't working? In gaefan's case, the fact that they can create new service accounts and make them owners suggests not, but it would be worth checking that you're not running into this case:

• You're logged in to console.cloud.google.com as your.name@gmail.com in one browser.

• You're logged in to console.cloud.google.com as your.name@company.com in another browser, and have access to the application there.

• gcloud auth application-default login pops up the login / authorization flow in the first browser and generates an Application Default Credential for your.name@gmail.com, when you really need one for your.name@company.com.

Answer 3

The remote_api_shell.py uses API credentials stored here:

~/.config/gcloud/application_default_credentials.json

You can rename that file and run command 'gcloud auth login' to reinitialize the login credential. This worked for several similar situations.

EDIT:

It may also worth to force the scopes using the command like that :

gcloud auth application-default login --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/userinfo.email'

You can find more details about the scopes here. They should be set by default, but it may worth to give it a shot.