Google Calendar API returning an 'HttpError 403' even with scope providing full read/write access?

Question

I'm trying to adapt the Flask example code at https://developers.google.com/api-client-library/python/auth/web-app to the Google Calendar API. Here is what I have so far:

# -*- coding: utf-8 -*-

import os
import flask
import requests

import google.oauth2.credentials
import google_auth_oauthlib.flow
import googleapiclient.discovery

# This variable specifies the name of a file that contains the OAuth 2.0
# information for this application, including its client_id and client_secret.
CLIENT_SECRETS_FILE = "client_secret.json"

# This OAuth 2.0 access scope allows for full read/write access to the
# authenticated user's account and requires requests to use an SSL connection.
SCOPES = ['https://www.googleapis.com/auth/calendar']
API_SERVICE_NAME = 'calendar'
API_VERSION = 'v3'

app = flask.Flask(__name__)
# Note: A secret key is included in the sample so that it works.
# If you use this code in your application, replace this with a truly secret
# key. See http://flask.pocoo.org/docs/0.12/quickstart/#sessions.
app.secret_key = 'REPLACE ME - this value is here as a placeholder.'


@app.route('/')
def index():
    return print_index_table()


@app.route('/test')
def test_api_request():
    if 'credentials' not in flask.session:
        return flask.redirect('authorize')

    # Load credentials from the session.
    credentials = google.oauth2.credentials.Credentials(
      **flask.session['credentials'])

    # drive = googleapiclient.discovery.build(
    #   API_SERVICE_NAME, API_VERSION, credentials=credentials)

    # files = drive.files().list().execute()

    calendar = googleapiclient.discovery.build(
        API_SERVICE_NAME, API_VERSION, credentials=credentials)

    # events = calendar.events().list().execute()
    calendars = calendar.calendarList().list().execute()

    # Save credentials back to session in case access token was refreshed.
    # ACTION ITEM: In a production app, you likely want to save these
    #              credentials in a persistent database instead.
    flask.session['credentials'] = credentials_to_dict(credentials)

    # return flask.jsonify(**files)
    return flask.jsonify(**calendars)


@app.route('/authorize')
def authorize():
    # Create flow instance to manage the OAuth 2.0 Authorization Grant Flow steps.
    flow = google_auth_oauthlib.flow.Flow.from_client_secrets_file(
        CLIENT_SECRETS_FILE, scopes=SCOPES)

    flow.redirect_uri = flask.url_for('oauth2callback', _external=True)

    authorization_url, state = flow.authorization_url(
      # Enable offline access so that you can refresh an access token without
      # re-prompting the user for permission. Recommended for web server apps.
      access_type='offline',
      # Enable incremental authorization. Recommended as a best practice.
      include_granted_scopes='true')

    # Store the state so the callback can verify the auth server response.
    flask.session['state'] = state

    return flask.redirect(authorization_url)


@app.route('/oauth2callback')
def oauth2callback():
    # Specify the state when creating the flow in the callback so that it can
    # verified in the authorization server response.
    state = flask.session['state']

    flow = google_auth_oauthlib.flow.Flow.from_client_secrets_file(
        CLIENT_SECRETS_FILE, scopes=SCOPES, state=state)
    flow.redirect_uri = flask.url_for('oauth2callback', _external=True)

    # Use the authorization server's response to fetch the OAuth 2.0 tokens.
    authorization_response = flask.request.url
    # import ipdb; ipdb.set_trace()
    flow.fetch_token(authorization_response=authorization_response)

    # Store credentials in the session.
    # ACTION ITEM: In a production app, you likely want to save these
    #              credentials in a persistent database instead.
    credentials = flow.credentials
    flask.session['credentials'] = credentials_to_dict(credentials)

    return flask.redirect(flask.url_for('test_api_request'))


@app.route('/revoke')
def revoke():
    if 'credentials' not in flask.session:
        return ('You need to <a href="/authorize">authorize</a> before ' +
                'testing the code to revoke credentials.')

    credentials = google.oauth2.credentials.Credentials(
        **flask.session['credentials'])

    revoke = requests.post(
        'https://accounts.google.com/o/oauth2/revoke',
        params={'token': credentials.token},
        headers={'content-type': 'application/x-www-form-urlencoded'})

    status_code = getattr(revoke, 'status_code')
    if status_code == 200:
        return('Credentials successfully revoked.' + print_index_table())
    else:
        return('An error occurred.' + print_index_table())


@app.route('/clear')
def clear_credentials():
    if 'credentials' in flask.session:
        del flask.session['credentials']
    return (
        'Credentials have been cleared.<br><br>' +
        print_index_table())


def credentials_to_dict(credentials):
    return {'token': credentials.token,
            'refresh_token': credentials.refresh_token,
            'token_uri': credentials.token_uri,
            'client_id': credentials.client_id,
            'client_secret': credentials.client_secret,
            'scopes': credentials.scopes}


def print_index_table():
    return (
        '<table>' +
        '<tr><td><a href="/test">Test an API request</a></td>' +
        '<td>Submit an API request and see a formatted JSON response. ' +
        '    Go through the authorization flow if there are no stored ' +
        '    credentials for the user.</td></tr>' +
        '<tr><td><a href="/authorize">Test the auth flow directly</a></td>' +
        '<td>Go directly to the authorization flow. If there are stored ' +
        '    credentials, you still might not be prompted to reauthorize ' +
        '    the application.</td></tr>' +
        '<tr><td><a href="/revoke">Revoke current credentials</a></td>' +
        '<td>Revoke the access token associated with the current user ' +
        '    session. After revoking credentials, if you go to the test ' +
        '    page, you should see an <code>invalid_grant</code> error.' +
        '</td></tr>' +
        '<tr><td><a href="/clear">Clear Flask session credentials</a></td>' +
        '<td>Clear the access token currently stored in the user session. ' +
        '    After clearing the token, if you <a href="/test">test the ' +
        '    API request</a> again, you should go back to the auth flow.' +
        '</td></tr></table>')


if __name__ == '__main__':
    # When running locally, disable OAuthlib's HTTPs verification.
    # ACTION ITEM for developers:
    #     When running in production *do not* leave this option enabled.
    os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'

    # Specify a hostname and port that are set as a valid redirect URI
    # for your API project in the Google API Console.
    app.run('localhost', 8000, debug=True)

Note that I've commented out some lines relating to Google Drive and replaced them with ones pertaining to Google Calendar.

However, if I run the web application using python app.py and navigate to localhost:8000/test, I get an error:

googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/calendar/v3/users/me/calendarList?alt=json returned "Insufficient Permission">

Here is the full traceback:

It seems from How do I get around HttpError 403 Insufficient Permission? (gmail api, python) that such a problem can be fixed by adding a more permissive scope. However, according to https://developers.google.com/identity/protocols/googlescopes#calendarv3, the Google Calendar API v3 has only two scopes, of which I've passed the one to 'manage your calendars'.

How can I get around this error?

Update

Following tehhowch's comment, I looked up whether I have granted sufficient scope to the 'Cleo' web application which the client_secret.json refers to. However, it seems that the scope is granted:

I'm now wondering whether the problem might be with the redirect URIs. Below is my client_secret.json, in which I've scrambled the secret_key.

{
  "web": {
    "client_id": "821409068013-unernto9l5ievs2pi0l6fir12fus1o46.apps.googleusercontent.com",
    "project_id": "cleo-212520",
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://www.googleapis.com/oauth2/v3/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_secret": "asdfasdfasdfasdf",
    "redirect_uris": [
      "http://localhost:8000",
      "https://app.hicleo.com",
      "https://staging.hicleo.com"
    ],
    "javascript_origins": [
      "http://localhost:8000",
      "https://app.hicleo.com",
      "https://staging.hicleo.com"
    ]
  }
}

Note that the redirect URIs don't include http://localhost:8000/oauth2callback; this is because this is an 'old' version of the client_secret.json I downloaded before I added in the the Google Cloud Console:

Perhaps it will work if I 're-download' the client_secret.json? I'm not really sure whether this is relevant though as I've apparently already granted access to the Cleo app.

Update 2

I tried going through the flow in an incognito window to see whether the issue was with any cached Google credentials. After logging in with the same Google account, I got a warning that the scope had changed:

Warning: Scope has changed from "https://www.googleapis.com/auth/calendar" to "https://www.googleapis.com/auth/drive.metadata.readonly https://www.googleapis.com/auth/calendar".

Isn't it a normal thing for the scope to change? Why do I get a warning 'error' instead of the page proceeding normally? And how can I fix this?

Answer 1

I ended up solving it by basically 'starting over again': I removed the access for the 'Cleo' third-party app, re-downladed the client_secret.json file, and went to localhost:8000/test again in an incognito window. After logging in, it seems that page indeed returns the expected JSON response:

I'm not sure which of these steps 'did the trick'; why did it not work with the existing access (if that was the problem)?