jwlon81
jwlon81

Reputation: 359

upload multiple files at once from google drive to google cloud storage

my objective is to identify some files in google drive and upload them sequentially to a google storage bucket. I am using a google 'cloud function' to do this and have already done tests to confirm that the connection is working properly.

The issue I have seems to relate to how I get the name of the file - it is returning a 'none type' value. Please see my code below

    import os.path
    from google.auth.transport.requests import Request
    from google.oauth2.credentials import Credentials
    from google_auth_oauthlib.flow import InstalledAppFlow
    from googleapiclient.discovery import build
    from googleapiclient.errors import HttpError
    from google.cloud import storage
    import io
    from googleapiclient import discovery
    from pandas.io.json import json_normalize
    import google.auth
    import re
    import logging
    
    # Confirming Oauth for Drive API# 
    SCOPES = [ 'https://www.googleapis.com/auth/drive']
    creds, project = google.auth.default(scopes=SCOPES)
    service = build('drive', 'v3', credentials=creds)
    
    # Confirming Oauth #
    storage_client = storage.Client()
    ## target Bucket #
    bucket_name = 'my-bucket'
    bucket = storage_client.bucket(bucket_name)

    get_bucket = storage_client.get_bucket(bucket_name)
    team_drive_loc = 
    'https://drive.google.com/drive/u/0/folders/xxxxxxxxxxx'
    team_drive_parent_folder ='xxxxxxxxxxxxxxxxxA'
    #bucket = storage_client.bucket(bucket_name)
    
    
    query= "name contains 'Customer' and name contains '2022'    "
    drive_files = service.files().list(q= query,
                                 driveId =team_drive_parent_folder,
                                 supportsAllDrives= True,
                                 includeItemsFromAllDrives= True,
                                 corpora ='drive',fields="files(id,name)").execute()

    for file in drive_files:

            source_file_name =service.get(fileId=fileId, fields="files(name)").execute()["files(name)"]
            bucket = storage_client.get_bucket(bucket_name)
            blob = bucket.blob("incoming/iri/IRI_Updates/Ongoing_Sales_Data/2022/" + 
            source_file_name)
            blob.upload_from_filename(source_file_name)
            logging.info('Uploaded {} ...'.format(source_file_name))

...And this is the error i get. If anyone can help me source the file name correctly and upload to the gcs bucket, that would be great

Exception on / [POST] Traceback (most recent call last): File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 2073, in wsgi_app response = self.full_dispatch_request() File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1518, in full_dispatch_request rv = self.handle_user_exception(e) File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1516, in full_dispatch_request rv = self.dispatch_request() File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1502, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) File "/layers/google.python.pip/pip/lib/python3.7/site-packages/functions_framework/__init__.py", line 171, in view_func function(data, context) File "/workspace/main.py", line 86, in iri_data_sync source_file_name =service.get(fileId=fileId, fields="files(name)").execute()["files(name)"] AttributeError: 'Resource' object has no attribute 'get'

Upvotes: 1

Views: 765

Answers (1)

Linda Lawton - DaImTo
Linda Lawton - DaImTo

Reputation: 116868

The error message is telling you the issue

'Resource' object has no attribute 'get'

its not service.get its service.files().get

source_file_name =service.files().get(fileId=fileId, fields="files(name)").execute()["files(name)"]
        

you actually had it right with service.files().list you just removed forgot it with the get request.

Upvotes: 1

Related Questions