Google Api Client Library for python doesn't return nextPageToken in Files: list method with query

Question

I need to get the list of all files and folders in google drive owned by a user. For some reasons file.list method doesn't return nextPageToken in response and I see only few results, since I can't go to the next page.

I have tried API Client Library for python and API explorer, but I receive only several records per user.

My code

users = ['user1@domain.com', 'user2@domain.com']
drive_array = []

if users: 
    for item in users:
        page_token_drive = None
        query = "'%s' in owners" % (item)
        while True:
            drive_result = service_drive.files().list(q=query, corpora='domain', includeTeamDriveItems=False, 
                                                      supportsTeamDrives=False, fields='nextPageToken, files(id,owners)', 
                                                      pageToken=page_token_drive).execute()
            drive_array.extend(drive_result.get('files', []))
            page_token_drive = drive_result.get('nextPageToken', None)
            if not page_token_drive:
                break

I expect to get id of a file and owners array for all files owned by the user

[
    {
        "id": "12344",
        "owners": [
            {
                "kind": "drive#user",
                "displayName": "User1 User1",
                "photoLink": "https://lg",
                "me": false,
                "permissionId": "1234556",
                "emailAddress": "user1@domain.com"
            }
        ]
    },
    {
        "id": "09875",
        "owners": [
            {
                "kind": "drive#user",
                "displayName": "User1 User1",
                "photoLink": "https://lh5",
                "me": false,
                "permissionId": "56565665655656566",
                "emailAddress": "user1@domain.com"
            }
        ]
    }
]

Answer 1

According to the documentation, to authorize requests to G Suite APIs, you need to use OAuth 2.0 and you basically have two options (or flows if you want to stick with the official terminology):

1. User authorization with a consent screen (e.g OAuth 2.0 for installed applications)
2. Domain-wide delegation for server to server applications (e.g. Using OAuth 2.0 for Server to Server Applications)

With the first option, once that the user completes the flow, you can only access to its resources. So if you want to list all the contents of drive for different users in the G Suite domain, you need to use the second option.

I also recommend you to use the python client pagination feature to manage the listing of files.

Here is a working example of option 1 (Python 3.6)

import os
import pickle

from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request

SCOPES = ['https://www.googleapis.com/auth/drive', ]
users = ['user1@domain.eu', 'user2@domain.eu']

# we check if we save the credentials in the past and we reuse them
if not os.path.exists('credentials.dat'):

    # no credentials found, we run the standard auth flow
    flow = InstalledAppFlow.from_client_secrets_file('client_id.json', SCOPES)
    credentials = flow.run_local_server()

    with open('credentials.dat', 'wb') as credentials_dat:
        pickle.dump(credentials, credentials_dat)
else:
    with open('credentials.dat', 'rb') as credentials_dat:
        credentials = pickle.load(credentials_dat)

if credentials.expired:
    credentials.refresh(Request())

drive_sdk = build('drive', 'v3', credentials=credentials)

# drive files API
drive_files_api = drive_sdk.files()

for item in users:
    query = "'{}' in owners".format(item)

    drive_list_params = {
        'q': query,
        'corpora': 'domain',
        'includeTeamDriveItems': False,
        'supportsTeamDrives': False,
        'fields': 'files(id,owners),nextPageToken',
    }

    # first request
    files_list_req = drive_files_api.list(**drive_list_params)

    while files_list_req is not None:
        drive_file_list = files_list_req.execute()

        print(drive_file_list.get('files', []))

        # pagination handling
        files_list_req = drive_files_api.list_next(files_list_req, drive_file_list)

If you run this, you will be prompted for authorization and the script will run on your drive listing files owned by other users and shared with you.

If you want to use the server-to-server flow with domain-wide delegation to list all the files (not just the ones shared with you), here is another working sample.

from googleapiclient.discovery import build
from google.oauth2 import service_account

SCOPES = ['https://www.googleapis.com/auth/drive', ]
users = ['user1@domain.eu', 'user2@domain.eu']

credentials = service_account.Credentials.from_service_account_file('client_secret.json', scopes=SCOPES)

for item in users:
    delegated_credentials = credentials.with_subject(item)
    drive_sdk = build('drive', 'v3', credentials=delegated_credentials)

    # drive files API
    drive_files_api = drive_sdk.files()

    drive_list_params = {
        'corpora': 'domain',
        'includeTeamDriveItems': False,
        'supportsTeamDrives': False,
        'fields': 'files(id,owners),nextPageToken',
    }

    # first request
    files_list_req = drive_files_api.list(**drive_list_params)

    while files_list_req is not None:
        drive_file_list = files_list_req.execute()

        print(drive_file_list.get('files', []))

        # pagination handling
        files_list_req = drive_files_api.list_next(files_list_req, drive_file_list)